Ok, first off, I STRONGLY advise you leave the voltage ALONE until you get very comfortable with overclocking.. for one, a .5 - 1 volt increase will almost certainly fry any chip.
Anyways, damaging components isn't real easy to do while overclocking, depending on what you are doing. If you're just bumping up the FSB, you ALMOST have little to worry about. Things to consider are PCI bus speed, and video card speed..I had to turn down my AGP bus speed because after OC'ing I had major graphical errors in games.
Always remember to go slow. Go maybe 3 - 5 MHz at a time, boot up, and check temps/take a benchmark test and make sure it won't die on you. When you find a clock speed you want to stay with, or after you've gone like 15 - 20MHz, you might want to run Prime95 for about 10 hours, and make sure it's stable there. If so, keep going..if not, back down a little.
Another thing you have to consider is your RAM speed. I dunno what you have for RAM, but I'm going to take a guess and say DDR400? DDR400 is only rated to run at a real clock of 200MHz (effective clock of 400MHz), so therefore you'll need to a) buy faster RAM (DDR500/PC4000 or more) or, b), setup a FSB:RAM ratio.
I won't really get into the FSB:RAM ratio thing because I don't really know much about it. In my BIOS, I don't have it..I just have DRAM clock where I can change the RAM speeds to slower settings, so I put it at DDR333 speeds to give me about 30MHz (.3GHz) to play with.
As for your temps - raising the FSB doesn't really raise the temps very far. At 2.0GHz, I run about 23*C, and at 2.25GHz I'm at 27 (both at idle, stock cooling). So it doesn't really raise it that much..but after you mess with voltage or really high clocks it will get hot.
Your temps are probally right around average, nothing to worry about. Most chips/mobo's/BIOS these days have alarms and shut-down temps anyways, so you won't hurt anything.
Oh and one last piece of info - say you raise the FSB too high for either the RAM or the CPU.. you don't have anything to worry about! If it doesn't boot/POST, try to get into BIOS but if you can't then just reset CMOS via the battery or the jumpers on the motherboard...just a little re-assurance for you.
I hope I helped, have fun.
EDIT: By the way, it is definitly worth it. I shaved ~11 seconds off a SuperPI test just by going from 2.0GHz to 2.2GHz, and I know my chip has a lot more potential.