My son just make one other suggestion - fence it. Even if it's a white low picket fence, create some barrier so it is more of intentional action for them to open gate and come onto property.

People don't understand there are rules about ratios in day homes which still apply outside. Or they don't realize that even if they stayed with their child, you would have an issue with them being present. Plus, sadly, they forget you are actually working. Unless you spell it all out, fencing might be your only option.