Hrm this is a long shot but it's going to come up sooner or later. It could be lack of power.
Okay what does your PC do when it has 4 slots, tell us how many beeps, blackscreen ect.
This is what I suggest depending on what it does.
-Rip any non essential components out (graphics card will depend on beep codes)
-If it's lack of power, your PC should boot without the load.
Also post what kinda power supply you have