I know scimitar propellers are used to reduce noise, but what is their downside? There has to be one, considering most propellers don't use that shape.
The biggest benefit of a scimitar propeller is the higher operating Mach number that can be achieved. Tip sweep delays the onset of shocks on the propeller and reduces drag at high subsonic speed. The noise reduction is achieved by reducing shocks on the blade tips at the same tip Mach number - supersonic props are known to be extremely loud.
The major disadvantage is the torsional moment created by the swept back tip. This can be somewhat mitigated by sweeping the inner blade forward which results in the typical scimitar shape. Still, in-blade torsion moments mean that the blade has to be built such that the deformation in flight will result in the desired blade twist. The use of stiff carbon fiber allows to keep this pre-twisting low.
That scimitar propellers have a high blade count is a result of their high operating speed: To keep the tip Mach number low, the diameter must be limited which increases the load on the single blade. Flying fast needs a lot of thrust, so the activity factor (power absorbed by the propeller per blade area) is high. To reduce the local Mach number on the blade, both chord and blade count are increased while the relative thickness of the blade's airfoil is reduced when compared to a conventional propeller. This results in a lower operating lift coefficient on the blades and lower supervelocities which again helps to increase the operating Mach number.
Molding of the intricate shape requires expensive carbon fiber
For it to dampen the noise it's best with 5+ blades (see 3.)
Which is hard to balance and too much weight for your run-of-the-mill 4-seater single engine plane
By the time the benefits (see @PeterKämpf 's answer) were realized, enter the jet engine for the big planes that would have benefited from it. You only see it in few turboprops nowadays.
The correlation between no. of blades and noise was mainly discovered from tests. Focusing on the cropped table below (see ref. for full table), the noise reduction potential is low for swept tips, and increasing the number of blades gets the full suite of penalties. Reduce the RPM by way of a gearbox for the fullest potential, and the same penalties apply again.
Metzger, F. Bruce. "An assessment of propeller aircraft noise reduction technology." Order 50 (1995): 50418D. https://ntrs.nasa.gov/citations/19960008053