3 quickest ways to quiet a computer down, IMO:
1. Change the power supply to one with a SINGLE 120mm fan.
2. Cut out all fan grills on case with a dremel, replace fans with quiet rated models, be sure to use rubber grommets instead of normal metal screws.
3. Replace CPU HSF with a quiet aftermarket one, preferrably with a fan larger than 80mm.
My computer used to be loud. And I mean LOUD. I could be out in the kitchen making dinner and I could hear the roar of it. I did these three things, and I can barely tell the difference from my computer being on and off now.
EDIT: Also, hard drives can be loud. Suspend your hard drive in a 5 1/2" bay with some type of lacing so it is not contacting any metal, like so: