I'm using the stock cooler on my E6600 without a problem...my load temps might be a tad high, but they never get anywhere close to that anyway. You just need to monitor the temps. It is recommended to get after market cooling, but like I said, it depends on the temps, and how high you want to go.
Basically, to overclock, you just increase the FSB. Stress test it for at least 8 hours. If it tests stable, you can increase the FSB more. Eventually, it will not stay stable, and you will have to give it more voltage. There are options in your BIOS to change those values.