If anyone else come into your house and treat your family this way, you be okay with it? For me, our home is somewhere my family feel safe. It our home first and I happen run business from it. If I was hairdresser with basement salon and someone come in and be aggressive to my family, I would not tolerate it. Just because this happen to be client or child, that not make it okay.
You say you quit your job to be home with your children but that not mean having accept poor behaviour or allowing your child be tormented and abused in her safe place.
You have two options :-
1. Terminate care
2. That child not leave your side since not trustworthy while you decide if he manageable or not. Bear in mind you intervene several time when he went to slap your girl. If you not able to keep him with you at all times, then he has to go.