I think you should fully disclose whether or not you take your kids out in the public sphere. The parents have a right to know where their children are and like others have said, if the parents aren't comfortable with that then maybe they aren't the right family for you.
I am in a similar situation to you in that when my group was smaller we were very active in the community - OEYC twice a week and library once a week. Now that my group is getting bigger (and it's winter) it's more challenging for us to get out and until I am a bit more established I don't have the funds to get the proper transportation (ie. a quad stroller). I tell all my parents just that - with a smaller group we were very active, are not so much now (although we do get out for walks around our neighbourhood every day and play in my yard), however in the future I hope to be able to attend those groups on a regular basis again.
I loved the playgroups because it gave me a chance to promote my business and interact with other adults!