In no way is this to blame you but it sounds like he is very jealous of your attention being given away to others. *not that it's an ok way to deal with it* Maybe try having special moments with just MOMMY so he still feels like you're not replacing him?
My daughter had a hard time too....one day about a month ago she was so bad and cranky, wanted to held all day - so I did! It was a "chill day"...she got her mommy-full and didn't act out again! I give her tons of hugs and kisses through out the day (sometimes the dck want hugs too and I'll give them but I don't kiss attack them like w/ my own).
Hang in there ((hugs))