The big advantage of radials at the beginning was their large frontal area, which meant they could be air cooled. An inline air cooled engine can run pretty hot on the rear cylinder. The bigger the engine, the more of an issue cooling becomes. There are some huge air cooled radials, in sizes that would be unthinkable for an air cooled inline engine.
As technology advanced, more complex water cooled engines became more popular. However air cooling was still a big advantage in military planes, due to the absence of a delicate cooling system that could be damaged, enabling radials to persist in this arena. Of course these days no-one would dream of putting a piston engine in a fighter.
Nowadays liquid cooling is almost universal for piston engines in general applications, with small aircraft being one of the few areas where air cooling has managed to hold out to some extent (see below.) In the absence of a military application that prefers air cooling, the large frontal area has become the radial engine's downfall, due to the large aerodynamic drag it produces. The other problem is that the valve gear is quite complicated, which means overhead camshafts and multiple inlet/exhaust valves per cylinder are impractical. This makes efficiency vs power to weight ratio a big issue as follows:
For an engine to have good power to weight ratio it must run at high RPM. To run at high RPM it must breathe effectively. To do this with just two valves they have to open wide. To stop them colliding with the piston, the compression ratio must be low, which limits efficiency. Hence most modern engines have more than two valves per cylinder to get around this problem.
Pratt & Whitney R-2800: 46L displacement, over 2000hp from an air cooled engine, by no means the biggest!
Excellent animation of the valve gear, which also tends to illustrate the difficulty of putting more than 2 valves on a cylinder. Also, lots of history (obviously.)
In World War II both Double Wasps (46L air cooled radial) and Merlins (27L water cooled V, licensed from Rolls Royce in Britain and built locally by Packard) were used by the American military. I once read a very good authentic World War II discussion about the tactics of when to use air cooled radial fighters and when to use water cooled ones. Unfortunately I can't find it. Here's one of the many discussions on the subject. Note that the Merlin had overhead camshafts and 4 valves per cylinder, like modern engines, whereas the Double Wasp obviously did not.
Modern air cooled engines
I am more of an engine guy than a plane guy, but I've just been looking at modern air cooled aviation engines. They are only available in much smaller sizes than typical WWII fighter engines. Typically they have 4-6 cylinders, too few for a radial, but enough to cause cooling problems on the rear cylinder in an inline.
The compromise is a flat (boxer) engine with 2 or 3 cylinders on each side. This also overcomes two other problems with radials: upper cylinders obstructing pilot view and oil collecting in the lower cylinder when stopped. You also have at least one of these problems with an inline, depending if you mount it upright or inverted.
Small aero engines can be either water cooled or air cooled. The designer has to ask: given the issue of reliability in aviation, does the weekend pilot really want an engine as complicated as the one in his car, with water cooling, cambelts and engine management, all of which could go wrong?
The air cooled engines I've seen (Lycoming) do only have two valves per cylinder, all driven from a common camshaft. (A simpler choice than the cam ring on a radial) I haven't seen an air cooled flat engine with overhead cams and multiple valves per cylinder, but it would be easier achieve than doing the same thing on a radial.