You can say you have given it a month trial and it isn't working out on your end and you need to go back to the 6pm pickup or whatever time will work for your family and no exceptions as in no being late.

It will probably mean they move on to another caregiver but if it brings your family peace that might be for the better.