Depends entirely on the error, and on the software that is being used. If the software has proper exception/error handling, then it will cache the errors, and dispose of them properly (the most common method of exception handling is to just dispose of the memory and start over).
Errors generally wont happen within the processor memory/cache or the cores, but they will generally happen in main memory. CPU memory is so small it's only used for small transactions. If a mathematical operation is performed wrong, then it will be passed into main memory and into the program; if a wrong answer is not handled, an exception is usually raised, and if there is no exception handling, then the program will generally crash, and the memory will either be freed by the OS, or it will be "stuck" and be considered garbage, and unusable.
Like the others said, cores don't necessarily restart, but they can go idle. They wouldn't restart (to my knowledge, anyway.. you can maybe find some more information deeper in some of the specific CPU architecture documentation).
If some sort of unrecoverable error were to occur on a core, then yes, there's a possibility that the system would restart.
An example of this sort of thing would be in this document; it's an example of how Intel Itanium CPU's work (error handling stuff starts on page 10 and page 11):
http://www.intel.la/content/dam/www/public/us/en/documents/guides/itanium-error-handling-guide.pdf