I will bring them in and let them play while I'm watching the old ones playing outdoors
Be really careful with this. If you have babies inside and older children outside, even if you are watching them like a hawk through a window that you never move from, in some provinces, this is unsupervised play. If you aren't within reach of a child, they are unsupervised and that's illegal.

Plus the liability issues if one falls and gets hurt are enormous.