I understand the need for a trimming mechanism, but why use a secondary control surface placed on top of the first, when you could just adjust the resting position of the aileron/elevator/rudder itself?
You have to have some way of holding the surface at other than where it wants to naturally trail in the airstream.
- Hold it yourself with control pressure. Becomes a pain after a while. Some very simple homebuilt designs don't have any pitch trimming device and they have only one trim speed where they will fly hands off, the trim speed more or less controlled by stabilizer incidence; to fly faster or slower you have to hold continuous pressure on the stick.
- Put a spring in the control circuit, a bungee, where the spring can be tensioned with a trimming device, usually a lever or screw device with a friction lock of some sort, to hold the continuous control force in the control circuit so you don't have to. Gliders do this, and some light airplanes and homebuilts for pitch. Works really well for aileron and rudder trim and lots of GA use bungees for the rudder.
- For pitch trim, make the horizontal stabilizer incidence adjustable from the cockpit. The elevator itself just trails when hands off the controls. The fabric Pipers and Cessna 180/185 family do this, but you have a screw jack device and a pivoting stab attachment etc., so it's a bit complicated. The fabric Pipers also combine the screw jack with a bungee interconnecting the elevator and stab so the stab applies a bit of elevator force when it moves, giving a bit more overall authority.
- Use an external servo device to apply a force to the surface. You have the air blowing past, so why not stick another moveable surface at the trailing edge that can move independently of the main surface, and apply an aerodynamic servo force. A bit more drag than a bungee trim or adjustable stabilizer, but much more sensitive and versatile. Much lighter and less complex than an adjustable stabilizer.
Probably 80-90% of the airplanes with manually operated flight controls use trim tabs for pitch trim because it's just the most elegant solution taking in all the considerations - light, simple, effective. Tabs are less common for aileron or rudder until you get up to airplanes over 5000lbs or so and you tend to see bungees or nothing, or little fixed bent metal tabs that only trim out to a single flight condition.