I've never been in the situation, but it sounds like you'd be we'll within your rights. Especially if your team of experts is advising the same.

If you have any contracts from the daycare it might be a good idea to look them over with a lawyer. Not that anything in a contract would excuse mental abuse of a child, but if the daycare was religious-themed, you may run into the issue of people questioning why he'd be enrolled there if it went against the daycares faith (if it were faith centered). But again, I'm not saying its excusable in any way.

Hope everything works out for you and your kids!