64 bit refers to the width of registers, memory addressing space, etc. One benefit is the ability to address more than 4GB of memory.
Wikipedia has an article on 64-bit computing with more details.
Edit: The advantages to more memory are that the operating system and programs have more virtual addressing space--16 exabytes (17.2 billion GB)--and, more importantly, that more physical memory can be added to a system and addressed, causing less swapping of virtual memory to and from disk.
The advantage to wider registers and data buses are that it is easier and faster to move the same amount of data around. An operation that required two or more registers can now be done with one.
So, performance is typically increased when software is recompiled for 64 bits.
A disadvantage is that wider data can mean more space taken by the same data. For instance storing the number 300 requires nine bits. If it's stored in a 32 bit integer, 23 bits are wasted. In 64 bit, that wastage becomes 55 bits. So, without retooling, a simple recompile to 64 bit can yield faster, but slightly more bloated software.
Edit: Also there are 64-bit technology pages here:
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…