I do my interviews in the evening. Never takes more than 1/2 hour. If the parent was dragging things out, I would let them know nicely that I had dinner to make or kids to get to bed or whatever, and tell them if they have any further questions to email me.
As for the references, once I've asked someone to be a reference, I don't let them know every single time that they might get a phone call. Just a general heads up when I'm going through interviews.