Did they have ANYTHING to say about the speakers or Printer that you were shorted on?
In my opinion, you should call them back, request to speak to a manager, and just explain that you are a very dis-satisfied customer, and you just want to recieve the product that you ordered and PAID FOR... Tell them that you can't send them just the PSU, cause you don't know how to take it out or something... Tell them you have a PSU phobia, and it would cause undo-mental stress to take it out...
Bottom Line, show them that you are DISPLEASED, and willing to give them a chance to correct it. You get A LOT FARTHER if you aren't a jerk also..