What scientific evidence is there about the effectiveness of pre-flight walk-arounds?

Question

As a student pilot, often on my walk-around, I get the impression that there are any number of hazards that could pass me by, simply because I can look at every part on the checklist but I wouldn't necessarily recognise a problem with that part. This idea seems more plausible when I speak to more experienced pilots of the same type, and each of them warns me about checking one thing in particular that's not on the checklist, because they once saw an accident that was caused by a failure there - and crucially, each of them warns me of a different thing.

At the same time, when I read the monthly accident report bulletins, there are several every month where the cause was some part failing in flight, and they either imply that the failure should have been caught in a pre-flight inspection, or they explain why it might not have been visible. In one local incident a few months ago, a pilot made a hard landing and thought he made an extra-careful inspection afterwards, but made a subsequent flight without noticing that both blades of the prop had been shortened and the firewall was crumpled. More generally, taking off with your pitot cover still in place is the stereotypical example of a bad inspection.

Pre-flight inspections seem very error-prone, on the face of it. What evidence is there about how error-prone they are, and what fraction of defects they detect? I'm asking for either scientific experiments under controlled conditions, or statistical analyses of aircraft grounded after unsatisfactory inspections vs. failures in flight. Turning the question around, you might instead answer about how much less safe we would be in GA or air transport if we didn't have pre-flight inspections at all. This related question is pretty relevant, but focuses on checklist items which are forgotten completely: I'm hoping for a more general answer which also includes defects that are not detected even if the checklist is performed correctly.

Answer 1

First off you are required by the FAR's to preform a preflight check,

Sec. 91.7

Civil aircraft airworthiness.

(a) No person may operate a civil aircraft unless it is in an airworthy condition.

(b) The pilot in command of a civil aircraft is responsible for determining whether that aircraft is in condition for safe flight. The pilot in command shall discontinue the flight when unairworthy mechanical, electrical, or structural conditions occur.

Don't forget the Preflight is also about making sure the aircraft is legally airworthy (VFR or IFR minimum equipment) as well as being physically airworthy.

Here is a brief from the FAA on the matter. It has at least one example of an accident that occurred as a result of a bad preflight. According to this Flying Magazine Article 15 accidents have been the result of a bad preflight in the past 10 years (from the publish date of that article). Here is another example as well. Clearly it has lead to accidents in the past but you may have an issue finding any data on times that it has found things and potentially saved lives (as that is often not reported to anyone but the mechanic).

Nasa conducted a study on checklist usage (preflight included) which is along the lines of what you may be looking for.

Here is another FAA article on preflights and has some more specific stats on what you are looking for.

Consider this: NTSB aviation accident data show that in the last 10 years, poor preflight inspections caused or contributed to 156 general aviation accidents and resulted in 41 fatalities.

You can compare that to the total number of other GA accidents for an approximation of how important it is in causing a crash. of When it comes to the FAA a great deal of analysis is carried out on accident data not necessarily preemptively. One of the other analysis issues here as mentioned in the comments is much of the reasons that planes get grounded does not get reported to any official authority (aside from the mechanic) so the data may not be public or easily accessible.

If we include preflight checklists and walk arounds in the same category. Following the Northwest 255 accident that was the result of misunderstanding/misusing pre-flight checklists the NTSB did do a study on preflight (and all checklist) procedures. You can find the study and results report here which includes some stats.

This idea seems more plausible when I speak to more experienced pilots of the same type, and each of them warns me about checking one thing in particular that's not on the checklist, because they once saw an accident that was caused by a failure there - and crucially, each of them warns me of a different thing.

This may be more a case of correlation over causation. If you are involved in aviation long enough (especially the older generation that has flown when laws and planes were much different) you see lots of things. Some of the issues they talk about may have been covered by AD's since they encountered them (if you are flying a common airframe like a 172 or PA-28).

As for not finding stuff this will come with how well you know your plane. As you start to fly the plane more and more (you should be training in one plane or at least only one type of plane). Eventually you will begin to know whats where, the POH can help with that, and you will be able to recognize if something is out of place.

I can speak from personal experience that I have delayed many flights due to things I have found on the preflight. I had a cracked cowling bracket, countless low gear struts that needed air, many fuel calls in the rental planes, a fuel vent issue, and one breaks issue.

Of course you will always have human factors present in manned flight.

Answer 2

This answer does not answer the question. Rather, I'll try to explain why I believe it is unanswerable (on this site at least).

---

Let $A$ denote having an accident (of any cause), and $I$ denote having a preflight inspection.

This question is basically asking the comparison of two conditional probabilities: $P(A | I)$ and $P(A | \lnot I)$.

Unfortunately nobody reports an event where $\lnot I$ is true, since you're never supposed to skip a preflight inspection.

---

Another way of looking at it will be, given a proper preflight inspection is done, there can be 3 outcomes:

1. A potential safety hazard was discovered. The flight was grounded. (We never know if it would have caused an accident)
2. No hazard was discovered, but the flight ended with an accident anyway.
3. The flight proceeded safely.

Note that $P(A | I)$ would be (2), while $P(A | \lnot I)$ would be (1) + (2).

However, (1) is not reported. Aircraft maintenance logs are generally not sent to an aviation authority, unless an investigation has opened on that airframe. This precludes scanning a database for (1).

Another word for (1) is how many accidents have been prevented by a walk around.

Airlines can reasonably have this data. They would know how many mechanic inspections are called because their pilots found something unsatisfactory in their walk-around. However I doubt such data is ever published (except for occasional news report), after all, airline pilots are not supposed to find something during their walk-around. If they do, it shows the airline's maintenance is poor.

Answer 3

Nearly every pilot publication, whether a book on the subject or an aircraft flight manual, will address the tropic. I can't speak to data driven scientific reporting on the subject, largely because much of what's caught on a preflight will never be reported (and thus omitted as data). I can speak anecdotally, from personal experience doing preflights, and having been flying for nearly four decades: preflights are critical. Post flights are also very important.

I've operated five airplanes that turned out to have cracked wings. In two of those cases, the wings separated in flight, killing all aboard. In the other cases, the cracks were discovered in time to prevent such events. I have found cracks, leaks, smoking rivets, fuel problems, maintenace errors, objects left in intakes, gear wells, fuel tanks, and other locations; I have found flight controls rigged backward, and once arrived to pick up an aircraft for a search and rescue mission only to discover the elevators missing.

Likewise on post flight, I've found parts of aircraft missing ranging from small exhaust stacks to 15' long fairings.

At this stage, you may not know what to look for, in that you may not catch everything; you may have more to learn, and you will certainly look for more things as you develop experience in general, and expertise with the aircraft that you fly. Even if you look for anything that doesn't belong, you're performing a critical task. Aircraft will talk to you: they seldom fail all at once, and will often manifest problems in various ways. It could be as simple as a smoking rivet; those streaking black lines that appear around rivets and metal seams, and which are really evidence of fretting corrosion and loose or misfit or damaged components. Aircraft talk to you; a good preflight is about learning how to listen.

Over the years I've found snakes in landing gear, people attempting to stow away, bird nests, and unreported damage. A minor fuel leak on a 747 turned out to be a cracked spar. A flap hesitation on a light twin turned out to be the spar cracked in three places: completely through.

I had a conversation one night in a combat area with a pilot who had just nearly run out of fuel. He'd taken an airplane at night on a reposition flight to another base, and discovered airborne that he did not have the fuel he anticipated. He attempted to use auxiliary tanks and found them empty. He was upset. He was angry at the pilot who flew it last, who did not ensure that the airplane was refueled. He was angry with me. I, in turn, was unimpressed with his preflight, non-existent, and his decision to take an airplane that wasn't yet fueled. A proper preflight would have prevented his near-disaster, and the risk it put him, the aircraft, and others in. Preflight preparation goes a long way to helping ensure a safe flight. I don't know the numbers behind it, but I can tell you from experience that it's a critical function, the importance of which should never be underestimated.

Answer 4

During the acceptance trials for WWII bombers in 1935, the Boeing entry crashed on takeoff as the gust locks for the rudder and elevators had not been removed. This accident was the catalyst for the preflight checklist. The preflight checks were done but obviously less than rigorously.

Boeing was allowed to use another plane of the same type, which won the competition and is known as the B-17, the legendary Flying Fortress. Yes, preflights and preflight checklists matter.

-Skip