I had the same problem after installing Bioshock today. I ran Bioshock when it came out and everything ran flawlessly, but this time I experienced the same Mouse movement shakiness. I messed with the settings for all my mouse drivers and Bioshock Mouse/Video settings for several hours because it was starting to really bug me, so I know EXACTLY what you're talking about.
(You can move around with the keyboard without moving the mouse and maintain a high FPS, but standing still and moving the mouse appears to lower the FPS. When actually for every movement it picks up, the cursor actually moves 3-4 pixels, which is playable, but does give you a headache after awhile. This is what you're talking about, right?)
I just fixed mine!; however, I don't know how much of this information will pertain to you, because my issue involved Logitech's Setpoint driver application.
Setpoint is Logitech's mouse application software which can override Windows Mouse settings and control sensitivity, acceleration, speed, etc. I first tried closing setpoint and using Windows default mouse settings with no third party mouse software. No result in Bioshock as the mouse still had shaky movement.
Anyways, I opened Setpoint and unchecked the option "Apply Game Settings when a game is detected." It basically forced SetPoint's settings, and then everything in BioShock looked amazing again with seamless mouse movements! The mouse movements changed with each pixel instead every 3 or 4.
So, I suppose it's your mouse driver.
FYI, I have a Logitech MX510, an MX518, and a G5 all hooked up to the same PC.
I had the same problem and fixed mine just now and saw that this thread was recent, so I registered to post this because I'm excited to finally fix mine
And now that I can play Bioshock... It's time for me to go to bed
.
Good luck.
gbdrue87@gmail.com