You do realize a video card has 2D and 3D clock frequencies, and when you are simply idling in windows in an environment that is not doing any heavy rendering instructions the core will revert to much slower 2D frequencies as the full power of the core is not needed
When you start up a game the core will revert back to 3D settings and speed up significantly