You are definitely NOT a failure, maybe you are doing way too much? I don't plan different things for the different age groups, I use the flash cards, finger puppets, feltboard games, etc. for the entire group and expect the babies to learn a wee bit each time as the months pass until finally they will be able to participate also. Same for crafts, the older children can do most of their own crafting and I do hand over hand glueing, stickers, etc with the babies but they do their own colouring and painting but I have them paint one at a time so I can monitor closely while the others play.

Here's an example. When I sing twinkle, twinkle little star to the group I open and close my hands for twinkling and today my 12 month old baby girl who has been with me less than a month started doing it. I was blown away. Repetition works, even with babies.