If you say something, then make light of it and tell the truth with a sentence or two, just as you did here. I wouldn't worry. If you were out for dinner and a daycare family saw you drinking a glass of wine would you be worried that they think you drink around the kids? You know? Your life is private after hours. Relax and have a good weekend.