video card overclocking is usually done via third party software which makes it nice and easy (ATi card - Download Ati tool , NVIDIA card - Download Riva tool)
Increasing the voltage on you CPU is what increases the temperature's so you are probably best to set the voltage manually and test from there. Your weak link is the stock cooler which you have, so I would find the default voltage for you CPU, manually set it to that, and maybe increase only slightly when you overclock it.
Otherwise your temps will sky rocket to what they were before (98-100c)