There should be an option called Load Line Calibration or VDroop (or something similar). This setting will either attempt to keep the voltage at a constant level or it will allow the voltage to fluctuate when the cpu needs more or less power. I recommend leaving it as you have it now, fluctuating.
Keeping the voltage constant subjects the cpu to more volts than it needs most of the time, which in turn makes it run hotter than it should at an idle or light load.
Don't max out your VCore, that's just a bad idea.
Not all cpu's overclock equally and some motherboards are better for overclocking than others. It sounds like your attempting to overclock your cpu beyond it's capabilities or beyond the capabilitiy of your present motherboard, or a combination of both.