the 32 and 64 bits correspond to the width of the data bus
1 byte=8 bits
32 bit bus can move 4 bytes at once
64 bit bus can move 8 bytes at once
by making the data bus bigger or wider the CPU can move bigger chunks of data per clock cycle and thus do more work in less time
someone above mentioned memory addressing, the address bus is separate and CPU's havent been "square" for a long time(address bus size=data bus size)
just for perspective:the nintendo orginal was 8 bit data bus and 16 bit address and video bus
when x86 PC's hit the market they had a 16bit data bus,and shortly after that the 32 bit machines came along, but they often had 32 bit data buses and bigger address buses, some of the latest 32 bit CPU's have 40+ bit address buses so they can address memory ranges bigger than (2^32)
At any rate, the bit number corresponds to the data bus width, the address bus is separate and varies in size
As far as 64 bit software, there isnt any,nothing really mainstream anyway,probably be 2-3 years before actual real 64 bit apps are commonplace.