We got asked once and I took a few days to think about it. I didn't want to get too personal and blur the lines between work and play.
In the end we did go because my own child and the daycare child were really good friends and playmates and I figured that we were asked because they had a genuine wish for the two children to get together.
It went well