Since they came yesterday and paid the fees owing plus I assume the late charge then they have actually done what you asked - paid up or no care on Wed. Which sort of negates your right to immediate termination - that should have been done on Sunday due to no payment before saying late charge would be added as in that gave them the option to stay and pay which they did.

If you decide to terminate then you are basically terminating and applying their deposit to the next two weeks. Just gotta hope they don't stay.