A lot of dad's aren't comfortable sitting on the floor. Normally when I do an interview we use the couch and I pull up a chair for myself. We are in the baby area though so I encourage them to put the child down on the floor to play and pull out a bin of toys for them. I like to see what the child does out of mom's arms.
Then if they move to the floor with the baby we all usually eventually gravitate down there too.
What was it about them that made you feel the most uncomfortable. Be sure that there really is something tangible and that it wasn't just parents not sure of how to do an interview - did they say how many they had done - or parents exhausted from doing interviews.
The fact they actually called back and want to place their child means no matter how you felt during the interview you managed to pull of friendly and competent so kudos for that. If you truly feel you can't work with them then ok but otherwise is it worth giving them the benefit of the doubt and having another meeting. This time you take full control adult to adult to work out the business side of things and more or less ignore the child part of it.