Yeah, if the HSF is dirty, that may be it... Also, if it's an older computer, the thermal paste may need to be changed.
There's also a chance that the temps are fine, and you have a faulty sensor.
If you feel safe doing this, open your case and feel the air that is coming OUT of your HSF (Most HSF fans suck IN, so you need to find where the air exits), after your computer has been on for like an hour, and it's idle. If it feels room temperature or slightly warmer, you're probably ok. If it feels warmer (comparable to breathing on your hand), there may be a problem.