I get attached. For some, it takes longer than others. I have one little guy in my care that I adore. He was the 1st child in my daycare. I often reflect back on how little he seemed when he started here and how well he's grown, how well he talks. I get extremely excited about all his accomplishments. I miss him when he is away for a while. Even my husband adores him. I think it has a lot to do with how well he took to us and how much he looks up to us. He reminds us so much of our own children and easily blends in to our family. He is a very easy child to care for.
On the flipside, I have a little girl that had trouble adjusting to being here. While she looks to me for comfort, she is very jealous of any affection I show to others. She doesn't do well with the daycare routine and has a hard time adjusting to new children coming in. She takes a lot of energy! lol. It took a really long time for me to feel any attachment to her, but it is there. I'd be sad if she left, despite all the energy she consumes. When she feels comfortable and secure, she is a very funny and happy little girl. She does a lot of cute things that remind me of me when I was little. Maybe that is why I have the attachment now. It took a long time to get there though.
I have 2 new dc children and I do not feel attached to them. I feel very indifferent. It is hard to say if I'll ever feel it. I guess time will tell.
Maybe it will just take you a little while to get to that feeling. Maybe it will take their absence to realize that it was there.

No matter how you feel about the children, I do not feel that you are a horrible person. Some of us become attached and some of us do not. As long as you can act like you adore them and make them feel secure, that's all that counts!