at the same time i felt guilty that if he is truly that hungry that I don't feed him.
Logic tells you that he's getting more than enough to eat, especially when you remember that he's likely eating more than you are, as a grown adult. Plus, if it helps you feel less guilty (and you have no reason to feel that way), consider that we provide lunch and two snacks. We are not here to provide the main meal of his day.

I hate to sound cynical, but with what you said and Dad's comments, I suspect this little boy is just having a small snack when he gets home before bed because he's full from gorging all day. That's quite the grocery saving for Mom when you consider her day care fees savings too. She's on to a winning formula all around.