There are a few reasons that the APU is located in the tail in most modern aircraft.
As previously mentioned, the space in the tail is otherwise mostly unused. Aside from hydraulics and such for the control surfaces, there isn't much use for the space. Most aircraft don't have any other spaces like that. Alternate locations like the wing root are generally used for fuel tanks, landing gear, air conditioning packs, and other systems. This area is also outside the pressure vessel, which reduces the number of ducts and such that need to pass through the pressure vessel wall.
Another important benefit is fire protection. Having the APU in the tail places it as far away from the cabin and fuel tanks as possible, and being outside the pressure vessel makes it easier to isolate from those areas.
Having the exhaust exit the tail prevents it from interfering with any other parts of the airplane.
Not having it next to anything else also makes it easier to access for maintenance, generally through doors on the underside of the tail.
The drawbacks of having the APU in the tail include the need to run bleed ducts, fuel lines, and power supply wiring all the way back there. Also, the hydraulics and controls in the tail must be protected in case of a problem with the APU.