Go to Bruma East/South Gate. Take the first left after you enter, the front doors to houses are on your left. Keep going until you reach the first house on your right (more like a shack) which is Ongar the World-Weary. He has 100 lockpicks ($6 each w/o haggling), that replenish in his inventory each day.
EDIT - If hes not home just hang out for a couple hours (wait "t")
EDIT - If hes not home just hang out for a couple hours (wait "t")