Did this just the other day.
As peterhuang913 said you need to set the hard drive you are putting in as slave (secondary) and the one already in the pc as master (primary). Have a look on the front of your hard drive, next to the IDE pins is a group of 9 pins. I don't know the exact name but i called them jumper pins.
There should be a little connector on the jumpers. Take this off and consult the printed information, usually on the top of the hard drive or in a manual about which pins you should join together with this. This tells the hard drive what mode it should be in.
Your motherboard probably does only have one IDE . The IDE cable connecting the original hard drive to the mother board should have another IDE plug. You should connect the master using the plug at the end of the cable and the slave using the plug lower down the cable. Make sure to connect the four pin power connectors as well.
Boot up your pc and the two IDE devices should be detected and your pc will boot to windows. From my experience IDE can be temperamental. If it doesn't work check the jumpers and IDE connections and try again.
I'm not sure if this is correct but windows can only recognise hard drives formatted on earlier or current operating systems. For example if you are running windows 98 it will not detect a hard drive formatted in windows xp. But windows xp will recognise drives formatted in windows xp, 98,95 etc. Don't take this as gospel though as i'm not 100% sure about this.
