If it's not enjoyable to be outside, then we don't go. That's my standard, lol.
I get the kids out at least once a day, but if we walk out the door and the wind takes our breath away, then we play on the deck for a couple minutes and then head back in!