I finally figured out my problem. If I do the "quick reply" at the bottom of each page, it does not send me an email whenever someone replies. However, If I hit the "Post a reply" button and submit my reply there, I get an email. And yes, I have auto notification turned on, its just like it doesn't get activated when I do a quick reply.