Did you build this computer yourself or was it bought manufactured?
I have never myself come across such a faulty power supply, or one that would be as messed up as you think it may be..
Do any STOP error's come up in Windows indicating specific problems? Please reply with the version of Windows you are currently on.
Is the BIOS updated?
You have to consider that psu's are fairly simple parts..I'm just not sure if that would be the problem, especially considering the fact that when you to a plain restart there are problems... It may be your motherboard. Feed us some more specs and we will hopefully be able to help you figure this out.
Intel P4 2.8GHz H/T
Windows XP Professional