never skimp on the PSU... for other components there's other options that give just as much performance with tweaking... PSUs won't.
the case is usually that last thing people think of when building a system on a budget, and usually seems like it leaves the most to be desired (ie most people end up getting an Antec 300)
I have the WD Caviar SE16 640GB drive that I love, but if I were getting a new one right now it would be the WD Caviar Black. The 500GB is $75. Probably not necessary for a budget rig like this, so I would look at either the GP or the Samsung above.