Are there green looking thumb screws on the back of the case? 2 on each side? If so pull on those and turn. Then slide the side of the case off (the left if looking at it from the front).
Unplug the 2 cables (one power and one going to the motherboard). Once you get those off, (after you turned off the computer and unplugged it from the wall) you'll have one of two options. Most older compaqs had tool less design. There might be a gree bar by your floppy/cdrom drives. Slide it up and pull out the old floppy. Slide in the new one and re-attach the floppy and mobo cable. Push the green bar back down and you should be good to go.
If you don't have the green bar, then you have screws. If this is the case do the same as above but take off both sides of the case and remove the screws holding the floppy in place.