I doubt a cheap 4gb type hard drive is going to yield decent performance compared to leaving the swap on a fast but large hard drive that you are booting off of. Most 4gb (or in that range) drives are 5400rpm, have little to no cache, and in generally are pretty slow.
Desktop machine: 2 x Opteron 246, Asus K8N-DL, 2GB PC3200 ECC Reg., XFX GeForce 6600GT, 74gb WD Raptor, 2 x 19\" LCDs, Windows XP x64
Server machine: Intel P4 3.0GHz 2MB EM64T, ECS i865pe, 1GB PC3200, 36gb WD Raptor, Windows Server 2003
Laptop: Dell Inspiron 9100 (Intel P4 3.2GHz 1MB Prescott, i865pe, 512MB PC3200, Mobility Radeon 9700, DVD+R/DL Burner), Windows XP
Linux: P3 450Mhz, 386MB ram, Slackware 10.1 (Running mySQL/Apache)