A simple breakdown of what Virtual memory is.
Virtual Memory is a temporary storage for all data that cannot fit into your physical memory. If you have 512 MB of memory and you're running a program that takes 380 MB and want to start another program that takes 250 MB, it wouldn't all fit into memory right? so it takes all the data in memory from the first program and puts it onto the hard drive as virtual memory and then loads the second program into physical memory. It then swaps the data back and forth as needed. This way it can keep programs running if there's not enough physical memory.
Now your hard drive is much slower, so things won't run as fast from virtual memory. More physical memory means less need for virtual memory, which means your PC has to wait on the hard drive less.
So yes virtual memory is good to have (Usually set it to 1.5-3X the Physical Memory), but it does slow the PC down if it's used to much.