32bit operating systems can only address 4GB of memory IN TOTAL, this includes video memory etc
So when you have 4GB minus video card (512MB) you see ~3.5GB or 3.2GB depending on what else is using memory bandwidth.
Vista "uses" memory in a different way via Superfetch, Superfetch "holds" recently opened programme info in your memory which "reserves" a variable amount of memory. This may be the reason your "performance app" is reporting 1.5GB. Check your memory via Task Manager for a better reading.
To totally utilise 4GB or more of memory you need a 64bit operating system
EDIT: Vista has matured nicely with SP1, I don't think I'll be going back to XP.