I've been having the same problem.
AMD Barton 2500+
400W power supply
I connected only the power and power switch cables. Whenever I hit the power switch, the system always runs for about 5 seconds and then powers down. One interesting thing is that if I remove the CPU, the system stays on. I connected some speakers and heard the "No CPU" voice announcement, but there is no beeps/announcement when the CPU is in.
I tried another CPU (AMD 1600+) and get the same results. Also, this is the second motherboard that is doing the same thing (I returned the previous one). I also tried a different case.