Beyond the deposit - you indicated this was sort of the biggest of the red flags but that there were other concerns. If you really feel this family is not for you then best to cancel the interview and save you all the time. I would just email them and say after careful consideration there are some things about the care required and our professional relationship that concern me - I would spell a few of them out which gives them a chance to counter if they wish in case it was all a misunderstanding - and therefore I have decided that we are not a good fit and that we should both look at different care arrangements. Therefore I would like to cancel the interview that was to take place Friday night.