Great, thanks guys, you saved me a lot of worry. I took the side panel off of my case and put lots of cooling on it for now (until I get my 120mm side fan fixed) and I made sure that my room is extra cool. After a bit, I removed my cooling units, with a cool room just to see. FANTASTIC. With my room much cooler, my CPU temps dropped to about 35c idle and 45c full load.
I can't believe I didn't think of room temp. I guess I just figured that the fans were supposed to do such a good job, that it slipped my mind completely. I guess either that, or I just didn't think it would matter THAT much. I feel dumb... oh well, it happens to the best of us.