Personally I think it depends on the type of daycare you have. Some home daycares are run under the premise that the care provider is giving her 100% to the children during daycare hours. Not chatting with friends, cleaning their house etc. Other home daycares are done in a manner where the care provider is doing household tasks during daycare hours while still caring for the children.

I personally would be unsettled by having my child in a home with different men in it during daycare hours. It is one thing to have a live in spouse around during daycare hours (as I would meet them before signing a contract) but it is a bit different when it is someone that the parents haven't met.

I don't know how you should proceed. But I would consider how to think the parents would respond if they showed up unannounced to get their child and he was there. That should be your starting point for figuring this out :-)