I'm sure it's possible to use servo motors to get precise control of the ailerons, elevators or the rudder. But still, hydraulics are widely used instead.
Hydraulics do have a disadvantage that a damage to the hydraulic pipes can cause loss of control. Even with redundancy, there are cases where all three hydraulic lines get damaged at the same time and there's complete loss of control (can't remember the example). I think in some smaller aircrafts servo motors are used. But why not in the larger ones. Is it because of the torque required?
What are the issues with using electric motors? If used what are the safety concerns? In terms of safety, how different are they from hydraulics?