Well yes it would.
The heatsink is made of metal right, so the heat from the processor surface is conducted onto the base of the heatsink, and out onto the fins of the heatsink where the air moving past removes that heat as the ambient air temperature is lower than the heatsink. The heatsink will pass heat to the air particles passing by. The aim is to keep the surface of the heatsink at the same temperature as the air. So basically the heatsink is passing the heat from the processor to the air.
When you take the heatsink off, the processor has nowhere to pass the heat to, as it doesn't have a large surface area to dissapate the heat. So instead of heating up the heatsink to take the heat away, it heats itself up very fast.