There are two extremes:
- No augmented controls, i.e. most normal aircraft. Pilot controls control surfaces and engines directly.
- Heavily augmented controls (left, right, up down shifts exactly like that), like quadcopters. Pilot gives command how the aircraft should move that is translated by algorithm into control surface and engine inputs.
At either of the two extremes, stuff is decently well understood. It's when you go in between that it starts becoming problematic.
Augmented Controls
There is always a conflict between a beneficial change making flying easier for the pilot and a greater level of abstraction from the flight controls. One place where this is particularly prevalent is on Airbus aircraft, where misunderstandings between the pilot, interface, control and aircraft have lead to multiple accidents.
Simon's point here- what do you 'want' you control surfaces to do- is very relevant.
Dynamic System
Helicopters are incredibly dynamic. For instance, a sideways gust will push the helicopter to the side, but that would also result in a disproportionate rotation on the whole airframe caused by the surface area of the tail rotor fin. Yet the pure sideways (not moment) force from increasing the tail rotor power is uncompensated and would require main rotor adjustment again to stay in position. Any pilot response and input would have to be handled appropriately to prevent pilot induced oscillation.
Stability Augmentation is used, for instance on the EC135:
The Auto Flight System is hierarchical in concept and on G-IWRC comprised a three axis Stability Augmentation System (SAS) and an autopilot. The SAS consisted of a Pitch and Roll SAS (P&R SAS) and yaw SAS. The helicopter was also equipped with a pitch damper. These systems are used for stabilising the attitude of the helicopter about the longitudinal, lateral and yaw axes by applying limited authority inputs to the main controls.
The SAS system is designed for ‘hands-on’ operation, which means that the pilot must provide control inputs through the cyclic control and yaw pedals in order to control the attitude of the helicopter. The SAS is automatically activated during the start procedures and can be disengaged by pressing either of the SAS DCPL
switches located on top of each cyclic stick grip. Re-engagement of the SAS is through a four-way switch on the cyclic grip, labelled P&R/P – P/y RST. Eurcopter EC135 Accident Report
Pilot Role and Regulatory Aspects
I've worked on UAV systems that go 'the whole way' on control automation. I think part of the problem is the conservative opinion on the role of the pilot. Airbus have become heavily criticized for their design choices that are supposed to make it easier for the pilot to fly. Augmenting flight controls is risky business.
Changes are difficult to implement and testing expensive in the aviation sector. We could ultimately remove the pilot and whoever onboard (doctor, nurse) could just select a spot on a map as a landing site. By going to the stage of making push up/down/left/right... controls as you suggest, we have created fully automatic flight controls. It is (entirely) technically feasible, since that's what the autopilot does. Changing how active control inputs are interpreted from a use, licensing, training and execution standpoint is a whole different story.