Dream, I would do exactly as you do, ask who had it first and usually children are honest and tell you who had it first. If they don't then they both lose it and I put it up high. I'm always saying 'we don't take the toys our friends are playing with', ugh! Thankfully I just have one boy who does that.