There is nothing wrong with inviting one child - the same age as your son and not the younger siblings. Now that he is school age - he is inviting his friends from school. In this situation I would word the invitation that your child would like to invite his friend child xxx to his birthday party (bring swimming gear). However, in fairness to child xxx younger siblings if you would like to bring them to the party and stay with them to supervise you are all more than welcome to come. Please let me know how many will be attending so I can plan the food accordingly.

Then that lets you off the hook for babysitting the little kids while you are trying to provide a fun time for your son.

I do like the idea of hiring a lifeguard for the day and that would bring peace of mind to all families.