I'm 100% the same way even though you know 100% during the interview if they’re going to go with you or not. You also know if you’re a good fit with them too. It just sucks, as we want to be the ones to say no! LOL
Don't worry, as these people would never ever be your friends either. I just hate opening my home to complete strangers to walk through its so freaking private yet public! LOL