There are two approaches I've used for ARM/Linux tools. The easiest is to download a pre-built tool chain directly.
Pro: It just works and you can get on with the interesting part of your project
Con: You are stuck with whichever version of gcc/binutils/libc they picked
If the later matters to you, check out crosstool-ng. This project is a configuration tool similar to the Linux kernel configuration application. Set which versions of gcc, binutils, libc (GNU or uCLibc), threading, and Linux kernel to build and crosstool-ng does the rest (i.e. downloads the tar balls, configures the tools, and builds them).
Pro: You get exactly what you selected during the configuration
Con: You get exactly what you selected during the configuration
meaning you take on full responsibility for the choice of compiler/binutil/libc and their associated features/shortcomings/bugs. Also, as mentioned in the comments, there is some "pain" involved in selecting the versions of binutils, C library etc. as not all combinations necessarily work together or even build.
One hybrid approach might be to start with the pre-built tools and replace them later with a custom solution via crosstool-ng if necessary.
Update: The answer originally used the CodeSourcery tools as an example of a pre-built tool chain. The CodeSourcery tools for ARM were free to download from Mentor Graphics, but they are now called the Sourcery CodeBench and must be purchased from Mentor Graphics. Other options now include Linaro as well as distribution specific tools from Android, Ubuntu, and others.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…