You are being too nice. You aren't standing your ground. When a child develops a fever while at daycare, when the parents pick up I tell them (let's say it's monday) "well I guess I won't see Sally until Wednesday then because she won't be within the 24 hours to come back tomorrow. Hope she gets some rest tomorrow and do let me know if she will be coming Wednesday". I dont care if the child is feeling better on Tuesday, if she was sick monday she needs to be HEALTHY for a FULL 24 hours before she comes back.
And when a child develops a fever right around nap time, this raises a huge red flag for me because it's usually about the time that the meds wear off. You have no way to prove it for sure but I would be willing to bet that they dosed and dropped.