New client .....Would you keep them?

[apples and bananas]

For one day a week I would terminate, however, you've allowed them to do it twice now. So I'm not sure if it's fair to say to a client, "sure you can use your deposit" Or "sure you can pay when you pick up" and then terminate them based on improper payment.

You need to tell the client, in writing, pay is due at this time every week. Failure to adhere to the policy will end in immediate dismissal. In other words, if he shows up next week and doesn't pay then use the deposit and send them home with the letter.