Is the "deaths per passenger mile" metric more useful for evaluating air travel safety than "deaths per flight"?

Question

Commercial airtravel is often called the safest mode of transportation, based on comparing deaths per passenger mile. However, I'm wondering if, among all possible metrics, passenger miles are the best possible metric for airflight safety, to even allow meaningful comparison to the safety of, for example, car travel. By "best possible metric" I mean the metric that is most predictive of airtravel accidents/fatalities of the industry as a whole, not any individual flight.

According to this statistic, the overwhelming majority (79%) of all accidents happen while landing, takeoff, approach, initial climb, and taxi. These are all phases that every flight, whether long or short, have in common. Less than 5% of accidents happen during cruise, the phase in which most of the distance of a flight is covered.

This statistic says that, by my calculation, only about 30% of fatalities between 2011 and 2020 happened during cruise, so 70% are practically independent of passenger miles. These Airbus accident statistics for 2003-2023 broadly agree in terms of the ratio of fatal flight accidents by flight phase.

Based on that, would "deaths per flight" not be a much better metric to evaluate risk, because it is much more predictive of accident/fatality than passenger miles? And yes, I'm aware that you need a common metric to compare airplanes to other transportation, but what's the use of that comparison if that metric barely applies to airplanes to begin with?

Answer 1

Deaths per passenger-mile is the best we can get, and it is good enough.

It's crucial to realize that estimating the safety of a given journey is always a very crude approximation. Unless you have lots of robust statistics for the particular trip you plan to take, you're left extrapolating from data that's weakly correlated at best.

If you're considering whether to fly from Atlanta to Seattle or take the car instead, you would ideally want to look at the fatality rate on Atlanta-Seattle flights on a particular airline (because maintenance and crew quality varies), on a particular aircraft model, in the last five years (because the industry evolves a lot), and you would want at least 10 crashes in that data set to calculate at least a rough average rate. But we don't have any data remotely close to that, because accidents are so infrequent. So you're left extrapolating from data aggregated from trips between Bangkok and Jakarta and between London and Dubai, and that's already a really crude approximation.

Compared to that approximation, the issues you mention are entirely negligible. Most commercial airline flights are not much shorter than 1 hour and not much longer than 10 hours, so in the worst case, those really long flights will dilute the take-off/landing crash rate by a factor of ten. That's not that much of a difference actually, since there's several orders of magnitude separating air travel safety from passenger cars. Similarly, there's likely several orders of magnitude separating the safest flights from the least safe, so that's the general level of accuracy you can expect when aggregating them all.

Deaths per journey aren't too useful, because that metric either wouldn't work at all (if you considered all car journeys equivalent, 5-minute trips to the mall will dilute your statistics much more than long flights) or would require looking at only car trips from Atlanta to Seattle (not enough data for statistics again).

If you give up trying to compare modes of transportation and only want to compare different flights with each other, check out Arnold Barnett: Measure for Measure ("A statistician offers his perspective on the relative usefulness of different ways of measuring aviation safety"). It thoroughly compares and discusses various ways to measure safety. Table 1 shows that "deaths per passenger-mile" is still the best predictor of mortality risk per a randomly chosen flight, although the correlation coefficient is "just" 0.57.

Answer 2

I think that the "death per pax mile" is the appropriate measure because it includes achieving the goal of getting from A to B.

• Driving from the Indianapolis airport to the Atlanta, GA airport is roughly 11 hours (and involves navigating 4 major metropolitan areas).
• Flying from IND to ATL is roughly 3 hours.

Even if the "death per hour" rate were exactly the same for air & ground transport, I'd be exposed nearly 4 times longer when traveling by car, thus air travel is still a safer way to go.

However, since both trips cover the same amount of ground and both achieve the same goal of getting me from Indianapolis to Atlanta, "deaths per mile" does seem to give me the most useful metric.

If we were to use "deaths per hour" as the metric, I'd have to stop somewhere around Elizabethtown, KY when driving to keep the risk comparison the same as flying, and that wouldn't achieve my goal.

If one were to use the the "deaths per trip", how do you define "trip"?

• If I fly, but I have a stop in St. Louis, is that considered 2 trips?
  • It's certainly considered 2 cycles for the aircraft, in terms of maintenance and air frame life cycle.
• If I drive, would that be considered two or even three trips?
  • After all, I'm not going to make this trip on one load of fuel (a generic term to cover both gas/diesel and electric powered vehicles). If I drive until my fuel is depleted, is that one trip?

Answer 3

The problem is, that no other metric is significantly better, and all of them favour one mode of transport over all others.

For example, cars can be considered much safer, since most people survive a road-traffic collision, even without taking into account that a large number of collisions aren't reported/recorded precisely because there is little damage as a result.

So, how do you want to measure 'safety'? Is number of deaths even the best measure?

Answer 4

Deaths-Per-Flight Suggests a Perverse Strategy To Achieve Safety; Deaths-Per-Passenger-Mile Captures All Of Your Concerns

Deaths per passenger mile (D/PM) has a number of advantages to recommend it, and ultimately is superior to deaths-per-flight (D/F) for virtually all use cases, but especially the use case you want to use it for:

predictive of airtravel accidents/fatalities of the industry as a whole, not any individual flight.

Not all flights are of the same duration, nor is duration the only factor (is the flight mostly over the Atlantic Ocean, vs. a flight path exclusively over flat agricultural-use land, for example).

The good news is that, when analyzing air travel at the industry-as-a-whole scale, all of those factors end up in the same blender, reduced to a single statistical slurry: how many units failure are suffered per unit of service; how you define the 'unit of service' and how you define the 'unit of failure' really matters here.

Unit of Service

What's good about air travel? Why do we want it? If a plane flies from Boston to Los Angeles, is that a good thing? What if the plane is empty?

Since we're talking about 'air travel' we can assume that an empty plane going from Boston to Los Angeles is of zero value, but if we define the unit of service as 'a flight' which is what 'deaths per flight' does, we end up valuing the flight from Boston to Los Angeles as 'a flight.' If it crashes, we get a value of 2-deaths per flight (the folks up front) for that single case. This will bring the real number of 'deaths per flight' down as a consequence. If we, instead, value the movement of passengers, we run into a similar issue if a plane is only half-full for that same trip. A full plane produces twice as much 'travel' as a half-full plane. So the unit of measuring travel is not 'a flight' but is instead the movement of people from one place to another. We can do this on a per-capita basis to control for varying numbers of people on various flights, and as such we end up valuing planes that are more full (because they're giving us more 'travel' value by moving more people around). More people on a flight, however, means 'deaths per flight' will go up - because we're trying to get more butts in seats... including the seats that crash.

Similarly, a flight from Boston to Los Angeles gives us more 'travel' than a flight from Boston to Kansas City, holding passenger count constant. The folks who went to LA got further. Just as we expressed 'travel' on a per capita basis, we can express it as a form of distance by expressing it on a 'per mile basis.' Travel, thus, ends up defined in units of 'Passenger Miles.' Regardless of mode, the value of travel is a function of how many people get moved how far, more people = more passengers, longer distances = more miles. Now I can compare a flight BOS->LAX with 150 people to a flight BOS->MCI with 120 people on board in terms of relative safety. To the extent that both planes have the same chance to crash - as you propose - the BOS-MCI flight is giving us fewer deaths per flight - but we're getting less actual travel out of that flight. Deaths per flight fails to capture that. If we have the same chance to crash either way, clearly we should only want to fly when we're going further - in other words we're getting more travel value per unit of risk taken. The good news is that by measuring travel as a passenger-mile, we don't lose that intuitive calculus: the LAX trip produces more death, but it also gives us tremendously more value in terms of passenger miles, compensating us for the added risk.

Unit of Failure

Unit of Failure is the risk we're trying to estimate. It is widely known that a plane crash will ruin your whole day, and for purposes of pessimism we can simply say that everybody who's on a plane that crashes, dies. It's not true, but it helps us avoid underestimation of the risk. Just as we ran into trouble with trip length, however, we now have a problem that planes are varying degrees of full so a given crash isn't going to produce a consistent value for deaths. Moreover, we defined 'unit of service' as 'passenger mile' so we need to at least convert to crashes per that unit. This, we can do. The problem remains, however, than a flight that has two people on it is not the same as a flight that has 200 people on it - but both are 'a crash' so the unit of failure should probably be able to capture the severity of the event, which is a function of the number of souls aboard: deaths perfectly captures the aspect of crashes that we don't like, and it's conveniently in similar units to 'passenger mile.'

Thus, deaths per passenger mile is chosen as a metric because it allows the aggregation of data from jets and turboprops and piston craft; transatlantic and regional; charter and scheduled; red-eye and prime time; deadhead and fully-booked. That aggregation smooths out the noise from any of those specific factors - but also allows comparison between them for sensitivity analysis that could reveal, for example, if transatlantic flights were more dangerous (because crashing in the middle of the Atlantic involves a MUCH lower chance that you'll be rescued) or not (because, as you point out, cruise phase isn't where the bulk of trouble occurs).

Intermodal comparisons

Deaths per passenger mile also facilitates comparison to other modes of transportation on a normalized basis. One flight could move 200 people, requiring a minimum of 50 4-seat passenger cars to achieve the same result. Assuming 100 miles traveled, differences would wash, but if the plane traveled more than 600 miles we now need to talk about those cars stopping for the night, or proceeding to drive while tired (resulting in higher accident rates on a per car basis) - suppose a single overnight, 100 4-seat passenger car trips are now needed to achieve the same net result of a single airline flight. If both have a 2% chance of a given trip going badly, cars are now killing twice as many people for the exact same unit of production (200 people moved the given distance).

D/PM controls for all of that. Let's take 1000 miles for simplicity of math, and assume an automobile driver needs two days to make the same journey.

One flight of 200 people over a distance of 1,000 miles, with a 2% rate of 'everybody dies' = 200 x 2% / 200,000 passenger miles = 0.00002 D/PM.

50 4-seat car trips moving 200 people over a distance of 500 miles per trip, so needing two trips, with a 2% rate of 'everybody dies' = 50 x 4 x 2% x 2 / 200,000 passenger miles = 0.00004 D/PM.

As you'd probably intuit, if a car trip and a plane trip have the same chance to end in tragedy - the fact that I'd have to take twice as many trips by car to achieve the same result gets reflected in the D/PM - but NOT in the 'crashes/deaths per trip' statistic.

To add one more wrinkle, automobile crashes have much less energy involved and so have a much higher chance that one or more passengers survive a severe crash than an airliner disaster does. So a more realistic comparison is that the car trips have a 2% rate of 'serious crash' within which the average fully loaded 4-seat passenger car kills, on average, 2.75 of its 4 occupants (the rest suffer serious injury but ultimately survive).

Now we're not even comparing 'chance of dying' to 'chance of dying' if we look at 'chance of crash' as the statistic of interest, but D/PM corrects for this automatically.

In our 50x2 4-seat car trips for 1000 miles case, we expect 1 out of 50 (2%) to end in tragedy, killing 2.75 people (instead of 4). 50 x 2.75 x 2% x 2 / 200,000 passenger miles = 0.0000275 D/PM - much closer to, but still technically more dangerous than our hypothetical passenger flight.

D/F is subject to the same issues that 'chance of a crash' is in the final hypothetical: it's impossible to meaningfully understand if air travel is generally safe if I'm measuring 'success' as 'a plane flew from A to B.' An empty plane produces a flight, but has almost no ability to contribute deaths-per-flight if it crashes, killing the two people up front and no one else. The safety protocols likely to be recommended by a D/F metric used to evaluate the air travel industry is 'keep as many seats empty as possible so we get more flights.' That doesn't reduce the number of people who die, it just means we now also lose more airplanes doing it. D/PM escapes that fate because an empty plane, while it doesn't meaningfully contribute to the deaths-per part of the equation it also doesn't contribute to the 'passenger miles' part of the equation, because it's not usefully moving any people.

You get what you measure.

It is also equally important to note that if you value safety in terms of 'deaths per flight' instead of 'deaths per passenger mile' you indicate to airlines that they should pursue safety in those terms as well.

In other words: Fly more planes, with more empty seats.

The demand for travel doesn't change, however, so this produces an aviation system that kills the same number of people (same % chance that any given flight you're on kills you) but now involves far more planes. It keeps deaths per flight down by simply ensuring that any given crash involves fewer people. Same % of travelers killed by this strategy has several downsides: air travel costs more (need to hire more pilots per passenger, buy more planes per passenger, consume more fuel per passenger), and now we're also crashing more planes in absolute terms in order to achieve safety.

In public policy (my field) this is known as a perverse incentive structure. Deaths per flight, as a metric, produces more waste and as long as those planes are hitting at least one person on the ground, we're also getting an actual increase in death.

Measuring deaths per passenger mile, however, indicates the optimization strategy of: If we're going to risk a plane crash, let's make sure we're moving as many people as we can, as far as we can, so that we get the maximum value per crash we have to suffer. Yeah, that means more people die per plane crash - but for each of those deaths, the system as a whole is producing more travel which is the thing we actually want. The empty airplane flying BOS->LAX illustrates that it's not actually 'flights' we want. Nor is it 'airplanes.' What we ask of the system is 'move people a distance.'

So D/PM is optimized by filling large, efficient aircraft packed to the gills with the best safety equipment we can, operated by highly trained professionals who can be selectively chosen because we need comparatively fewer of them.

It's also optimized by preferring that flights operate in their safest mode for as much of the flight as possible. As you note, takeoff/landing are where the danger is: so for any given takeoff/landing we want to get as many passenger miles as we can. In other words, we want to use air travel for longer-distances, where they can justify the comparatively high risk on a per-flight basis - exactly the situation where the 'per trip' risk is amplified by limits of the passenger car, because one flight can replace many car trips for the same passenger miles value. (This is ignoring the fact that, because of the amateur nature of its operator population, car travel is absurdly more dangerous.)

What actually predicts accidents?

Because we're measuring aggregate system performance for a system that is used beyond a de minimis level, predicting 'will there be a plane crash' is trivially easy: Absolutely, there will be. Either metric tells you this, because there's enough planes flying around that it's not a question of 'if' anymore, it's a question of 'how many.'

Deaths-per-flight measures how many people are, on average, involved in a typical plane crash. All it tells you is how many people are in the plane.

Deaths-per-passenger mile measures how many people are killed as a function of the satisfaction of the purpose those planes fly for. This has the added value of allowing you to predict how many people will die if you have an estimate of the total distance people will need to travel by air. If you want to know how many plane crashes that will involve, you do still need to come up with an average human-beings-per-flight but the use cases for 'how many crashes' are few and far between (anticipated need for accident investigators is one potential case, however).

Answer 5

If you want to get rid of any dependency on length, what about death per journey or per hour? That should give a "more correct" number according to your assumptions. Those statistics should also be readily available.

I agree anyway with @MikeB that "no other metric is significantly better, and all of them favour one mode of transport over all others". For example, if you consider the death per journey as I suggested, you have anyway somehow to consider that a typical journey via airplane would be much longer than a typical journey by car... Maybe you could divide the death per journey via the typical journey's length.

Answer 6

The answer to your specific question between those two metrics is: No

No, "death per passenger mile" is not a correct metric compared to "death per flight". Why is that? Well, at leat two reasons justify this:

• First, dying at the end or the beginning of my journey from Canada to Australia does not matter a lot: in the end, I'm dead. But in one case, if the airplane crashes on landing, the death per passenger mile would be significantly better than in the case of a plane crashing during take-off. On the other hand, given that many accidents occur during takeoff, you could say in an hypothesis that all airplanes have accidents, that the ratio is X deaths to 0, which does not make sense.
• Second, the passenger mile number changes considering how many people are inside the plane: but whatever the number of bookings in the plane, you do not want it to crash anyway (unless it is a freighter plane where one might consider that safety rules could be lighter, and even that is not the case AFAIK)
• Third, since most accidents of airplanes' causes are not linked to the length of the trip, you should note take it into account. On the other hand as said above, one flight is dangerous because it passes through some specific moments such as takeoff*

*Please note that I'm not definitely stating that take-off is very dangerous nor is the most dangerous part of a flight: I am not entirely sure of that. I'm just taking it as a plausible example for this answer.