Reaching 4GHz is achievable, but, it's really really hard to have it stable at 4GHz with air cooling. Unless you leave in Alaska or really cold place.
And most likely to reach 4GHz, you'd need more than 1.5v.
3.6GHz is the average high for Q6600 G0s and that's plenty fast.
If you set it to 4GHz now, and set the vcore to 1.3, I can almost guarantee it won't boot.
Just do what you were doing before, going up increments at a time, and when it's unstable increase the voltage by a notch. And when you get to the 3.3GHz+ stage, you should increase the MCH and FSB voltages as well, for stability.