You're right in that the answer is quite complicated.
What you're looking to understand is ALUs and FPUs, which form the basis of pretty much every processor designed. They're made up of smaller circuits, which in turn are comprised of logic gates.
The pic below is of an 'adder' which is part of what an ALU is made up of.
I'd suggest reading up here, unless you understand the basics of binary logic and simple electronics though, it's not going to make much sense I'm afraid
Arithmetic logic unit - Wikipedia, the free encyclopedia
"As a result of all this hardship, dirt, thirst, and wombats, you would expect Australians to be a sour lot. Instead, they are genial, jolly, cheerful, and always willing to share a kind word with a stranger, unless they are an American." -- Douglas Adams