Looking at the sub $50 range bearing in mind, PP has hit the Nail on the head with a new PSU you could look in too something like:
EVGA 01G-P3-2615-KR GeForce GT 610 1GB 64-bit DDR3 PCI Express 2.0 x16 HDCP Ready Video Card - Newegg.com
Looking at the $50 too $100 price range, maybe something like this would help:
EVGA 02G-P4-2742-KR GeForce GT 740 Superclocked 2GB 128-Bit DDR3 PCI Express 3.0 Video Card - Newegg.com
I honestly think you might be wasting your time to be honest upgrading due too the fact the PC is a HP, I've always found over time that the best systems too upgrade and Maintain are the ones you build your self. Its like having a Dell System, they are a nightmare too play around with..
Have a look at the suggestions I've posted but as PP said you'll need a new PSU.
EDIT: Bearing in mind you might get reduced performance from the PCI Express series 3 Range of GPUs on a Mobo I'm presuming supports 2.0 ..