Shhhhhhh! Don't ever tell a man this, but their testosterone levels are falling just like our estrogen levels fall as we get older. It's normal.