Ok so your issue is still shutting down, or no?
Having not enough VRAM on certain games will only slow your FPS a bit or cause an annoying stutter. There are only 2 games right now (besides a heavilly modded Skyrim) that can do this at 1080p. The New Order with 8k setting applied (because of the way the engine uses texture streaming) and Watch Dogs. Wolfenstein is made to fill your VRAM regardless of size or setting because textures are streamed (it's an IDTech5 engine thing) so it's not supposed to cause adverse issues while being full. Considering I can play the game fine on my card it shouldn't cause issues on a 780ti.
If your CPU isn't going over 70c and your GPU isn't going over 90c you should have no issues revolving around heat.