I guess it depends... If I had made it VERY clear that the position may only be temporary, and that if/when the other family returns, the p/t position may not be available anymore ... then maybe. I would still feel pretty badly about it, though.
Is the previous family returning, FOR SURE? We all know that it's easy enough for them to say they are coming back, but when it comes down to it, they could change their mind. I would get some firm details hammered out, first. I have a family coming back that I thought/assumed would be full time again. Turns out they only want to do 2 days per week. It's ok, because I wasn't "counting" on them being full time. But still...