I would never have thought of this problem either...
Do you have this issue? I would absolutely explain that they are welcome at any time during dayhome hours, but that this type of "visit" is un-necessarily hard on the child.
I tell parents that they are welcome, and I have had early pick-ups, but nobody just popping in and leaving again. That seems like something that would be common sense on behalf of the parent, no? :P