Nope, I would not be okay with her waltzing in and "visiting". The next time her little one tries to say "Come in, Mommy!" I would say "Well, it's time for mommy to go, and for us to play with the other kids! We will tell her all about what we do at the end of the day, ok?"
If she physically tries to enter, then I would have to say something. "I hope you don't take offense to this, but I am actually not allowed to have parents come in and stay with their kids! Sorry, haha, I know it's silly, but it's really for the safety/comfort/security of the other kids and their parents. I wish I could, it would be nice to have some adult company, but what do you do, right? hahaha" If she tries to come in at pick-up, I would say "Oh, you stay there, so you don't have to TAKE YOUR SHOES OFF (lol) I will go grab her!"
I am guessing that she does feel a little bad dropping her off if she's not working, but as long as you guys have an agreement, that is her issue to deal with, not yours. She probably also has no idea that it's not ok, and just needs to be gently told so.
If approached properly, things should be fine. She will probably apologize ("Oh, sorry, I had no idea") and forget about it. If she does take offense.... ppssshh. Good riddance! I don't think any of my parents would have trouble understanding this point. It's for the good of everyone involved, since other kids also see someone's mommy coming to play and want theirs to stay too. Kids need to understand that we say goodbye to mom/dad at the door, and then we play, and see them at the end of the day (at the door!). My open-door policy means that you are more than welcome to pop by unannounced TO PICK UP YOUR CHILD. That's it. It does not extend to chit-chat when I am taking care of x-number of kids!