Basically everything that consumes power on a aircraft can potentially cause interference, short-circuts, or otherwise jeopardize the safety of flight and therefore must be switchable. Sometimes the switch is in the form a button, otherwise by a fuse.
There are several particular reasons that the transponder can be turned off.
If the transponder malfunctions, it may cause interruptions to all ATC surveillance in an area. There have been instances in the past that due to a fault in the transponder it was basically acting as a jammer.
In one particular incident it took a while before the aircraft that caused it was identified and after requesting the pilot to switch of the transponder, secondary surveillance was restored.
Another reason is that when the aircraft is at the gate, the transponder is switched of to reduce the amount of radio transmissions. 100 aircraft on the surface of a large airport can produce a massive Radio Frequency noise, which negatively affect radar systems. When taxiing, radar replies are useful for aircraft identification, hence the transponder is switched on at pushback or engine start.
The focus here and in the media on the transponder in relation to hijacks is a red herring. If an aircraft is hijacked, there are way more important things to worry about than an active radar transponder.
The fact that it is turned off means that hijackers already have access to the cockpit.