How will the main rotor torque be balanced if a helicopter doesn't have a tail rotor?
The torque in a helicopter is generated by the engine driving the main rotor in one direction, which causes the fuselage to spin in the other direction. The tail rotor shaft is horizontally mounted and hence creates it own 'lift' to provide anti-torque. The pitch of the tail rotor gives directional control.
Common ways to design a helicopter without a tail rotor:
Tandem rotors : Two main rotors mounted one in front of the other. The rotor discs spin in opposite directions. All power from the engines is used for lift. Example: CH-47
Transverse rotors : Two main rotors mounted side-by-side. Example: Kamov Ka-22
Co-axial rotors : Two rotor discs are mounted one on top of the other. The engine units power both rotors. This structure requires a very complex swashplate mechanism. Example: Kamov Ka-25
Intermeshing rotors : In this main rotor assembly, the two set rotor masts are installed at a slight angle to each other, in a transversely symmetric manner, so that the rotor blades intermesh without colliding with each other. Example: Kaman K-MAX
Tip jet : This is an interesting solution to the no-tail-rotor problem. Instead of driving the rotors with a turboshaft engine, compressed air is sent through nozzles installed at the tips of rotor blades. The engines effectively push against the air rather than the helicopter fuselage. The rotor spins just like a Catherine wheel. Example: Hiller Hornet
NOTAR : NOTAR - NO TAil Rotor uses a fan inside the boom to build a high volume of low-pressure air, which exits through two slots and creates a boundary layer flow of air along the tailboom utilizing the Coandă effect. The boundary layer changes the direction of airflow around the tailboom, creating thrust opposite the motion imparted to the fuselage by the torque effect of the main rotor. Example: MD-900
Refer to Wikipedia for more!
Another option to build a helicopter without a tail, is to use active flapping. Basically the idea is to not counteract the torque of the main rotor, but use a rotor concept that does not generate a torque.
At the Technical University Delft they applied this principle to build the Ornicopter (a mix of the words helicopter and ornithopter), a tailless helicopter. Flight tests have been performed using this prototype.
According to the university:
The Active Flapping
The active flapping of blades is the key for the Ornicopter concept. In this manner, the blade can generate a propulsive force to rotate itself, and hence the shaft torque is not needed. This results in a torque-less main rotor, i.e. the tail rotor can be eliminated.
Concerning the question on scalability posed in one of the comments:
I found a PhD thesis on the topic (download) which did a feasibility study on a Bo-105 scale ornicopter. The thesis says:
"The thesis proves that the optimal design for the Ornicopter as compared to the Bo-105 benchmark is characterized by a lower blade loading, increased rotor tip velocity and larger vertical fin size. This optimal design results in an enlarged flight envelope due to the reduced rotor stall area and improved yaw stability in forward flight.
Nevertheless, despite these improvements in the Ornicopter's flight envelope, there is a slight increase in required power when compared with the Bo-105 specification (approximately 5% at 150 knots).
To compensate for the higher profile power needed for the Ornicopter's optimal design, a larger rotor radius is required in order to reduce the induced power and keep the increase in the total required power to a minimum. This thesis may be considered as a first step in rationalizing the expectations regarding the Ornicopter's tailless helicopter design.
The thesis proved that this new concept shows a slightly poorer performance than that of conventional helicopters regarding power consumption in forward flight and service ceiling. This is disappointing since one of the assumptions was that the elimination of the tail rotor would also eliminate the power consumption associated with a tail rotor. "