I definitely would say keep it and put it in. It doesn't hurt to have one. A floppy drive will not mess with system performance, it only gets used when you actually access it, and only then does it use system resources.
I occasionally use it as a quick boot device, for quick file transfers (used it a lot in school because our high school wouldn't allow USB drives because they "were too risky for program stealing and viruses" [I think it was because the IT department was too lazy to actually bother with these things and just said "meh, whatever, just make it a rule"]).