50
$\begingroup$

Facts

On Airbus aircraft there are computers to secure the flight envelope, or to move the control surfaces. FADECs totally control the engines. Computers take decisions in place of the pilots, or even against their commands. Boeing aircraft have similar computers, even if the crew has more authority.

enter image description here
A330 electronic bay, source, photo by 'swiss_a320'

Safety implications

Being critical, systems are redundant and supervise each other to detect possible failures and isolate failed components. However a computer can still be developed from faulty specifications, or be wrongly manufactured, and a program can contains bugs. If the same defect is present on all computers on the production line, the purpose of redundancy may be defeated, as they wouldn't be able to detect erroneous behavior.

This is better stated in this article:

Because of the severe consequences resulting from a single point of failure, hardware redundancy is critical in DAL A systems. But if the aircraft uses a redundant architecture built with similar channels, that system will still be susceptible to common mode failures that can cause all channels to fail in the same way.

Question

What are the principles used in aviation to reduce the possibility for redundant computers to fail or to make the same errors at the same time?

$\endgroup$
  • 6
    $\begingroup$ 3 or 4, usually. But to answer your other question: if all redundant computers are manufactured or programmed in the same faulty way then the crew and passengers would indeed be screwed if it made through all reviews, auditing, and testing and be found on a operational product, which is why making such a thing and get it through the process and getting certificated is very expensive and time consuming. $\endgroup$ – user3528438 Oct 3 '17 at 21:34
  • 4
    $\begingroup$ Example of a bug making it through testing. Fortunately it’s quite a rare thing. $\endgroup$ – Notts90 Oct 4 '17 at 17:13
  • 5
    $\begingroup$ Perhaps they make the programmers go up on the first test flight. $\endgroup$ – Bob Jarvis Oct 5 '17 at 3:15
  • 1
    $\begingroup$ @Notts90: related issue regarding Patriot missile software bug in the 1991 Gulf War. $\endgroup$ – Bob Jarvis Oct 5 '17 at 3:19
  • 2
    $\begingroup$ There's two separate issues here, redundancy and dissimilarity. Level of redudancy is just a measure of how many systems you have commanding and monitoring, while dissimilarity is how separate and independent those system are. I'd suggest clarifying which one you're curious about by renaming the title something like "How dissimilar are redundant flight control computers" $\endgroup$ – Cody P Oct 5 '17 at 19:49
67
$\begingroup$

As far as Airbus is concerned:

  1. Each unit is composed of two dissimilar boards, one driving the output and the other checking it. Dissimilar means both different CPUs and chipsets (A320 uses i386 (Intel) and m68k (Motorola); newer models use different combinations, basically whatever was widely used at the time they were designed) and software written by two independent teams.

  2. There are fail-overs, two or three depending on the system (IIRC the unit reading the side-sticks is the only one with four copies).

  3. The two main axes, pitch and roll, are controlled by two different systems. ELAC controls elevator and ailerons, SEC controls horizontal stabilizer and spoilers. This is two completely independent chains including different actual control surfaces except for the side-sticks.

  4. A320 has (hydro)mechanical backup for pitch via the trim wheel and yaw via the pedals, utilizing yaw-roll coupling for roll. This works even with complete electrical failure. IIRC the backups on newer models don't though (because complete electrical failure has never happened).

$\endgroup$
  • 2
    $\begingroup$ Expanding on the dissimilar boards bit - idk about airbus or anyone else in particular, but I have heard of having three different systems for a decision - this means that if one is "faulty", it is not in agreement with the others, and the correct decision can be upheld, but you can't tell which is which with two systems. Is this right? $\endgroup$ – Baldrickk Oct 4 '17 at 11:09
  • 7
    $\begingroup$ @Baldrickk Yes, that's the normal reason for two-out-of-three voting: with two, if one is wrong, you can't know which one is wrong; all you can know is that they disagree. With three, if one is wrong, you can know because two are still in agreement. As long as not two out of three are wrong, three gives you far better ability to handle disagreements than do two. $\endgroup$ – a CVn Oct 4 '17 at 11:52
  • 4
    $\begingroup$ "because complete electrical failure has never happened" >> You'd think aircraft manufactures would have learned about Murphy's Law long ago. Ask Al Haynes about "impossible" failures. en.wikipedia.org/wiki/United_Airlines_Flight_232 :-/ $\endgroup$ – Shawn Oct 4 '17 at 14:03
  • 3
    $\begingroup$ @MichaelKjörling See: Minority Report $\endgroup$ – SGR Oct 4 '17 at 14:45
  • 4
    $\begingroup$ @Baldrickk, In general yes, but it never applies to Airbus. All airbus systems have one unit making the decision and another one system verifying it. And if the verification fails, the two-system unit is declared faulty and a fail-over approach is used. $\endgroup$ – Jan Hudec Oct 4 '17 at 19:49
37
$\begingroup$

Redundancy is not only achieved by multiplying the computers, but also by diversifying them. On Airbus airliners, two different computers are used (one with Intel chips, the other with Motorola chips in case of the A320) and software is written twice, one for control, the other for monitoring, by two teams which are not allowed to interact.

To cite from chapter 12 of The Avionics Handbook:

Despite the nonrecurring costs induced by dissimilarity, it is fundamental that the five computers all be of different natures to avoid common mode failures. These failures could lead to the total loss of the electrical flight control system. Consequently, two types of computers may be distinguished:

2 ELAC (elevator and aileron computers) and 3 SEC (spoiler and elevator computers) on A320/A321 and,

3 FCPC (flight control primary computers) and 2 FCSC (flight control secondary computers) on A330/A340.

Taking the 320 as an example, the ELACs are produced by Thomson-CSF around 68010 microprocessors and the SECs are produced in cooperation by SFENA/Aerospatiale with a hardware based on the 80186 microprocessor. We therefore have two different design and manufacturing teams with different microprocessors (and associated circuits), different computer architectures, and different functional specifications. At the software level, the architecture of the system leads to the use of four software packages (ELAC control channel, ELAC monitor channel, SEC control channel, and SEC monitor channel) when, functionally, one would suffice.

$\endgroup$
  • 5
    $\begingroup$ Famously the 777 has three redundant computers created by different teams/companies in different programming languages. $\endgroup$ – Martin Schröder Oct 5 '17 at 12:08
  • 1
    $\begingroup$ @FreeMan: The Eurofighter uses also the Motorola 68000. Barring some midlife upgrade, those have to be available for the next 30 years ore more. To keep the last one flying will probably require to steal the chips from a museum. At least the more modern Airbus computers use the PPC. $\endgroup$ – Peter Kämpf Oct 5 '17 at 16:55
  • 4
    $\begingroup$ @FreeMan: Those are military grade chips which have passed a very long approval process. Part of the process is to make sure that somebody will be able to manufacture those chips in decades to come. Of course with plenty of government financial support. These are really juicy contracts, so the manufacturers will not die out or lose interest. Hint: It's not alone the toilet seats that are more expensive than their equivalent in massive gold. $\endgroup$ – Peter Kämpf Oct 5 '17 at 18:39
  • 2
    $\begingroup$ @FreeMan Even if the cost of a mil-grade 68000 was a "ridiculously high" number like $10,000 per part, that would be negligible compared with the cost of certifying a newer chip design. For example even one hour of flight test time would cost more than $10,000 just to fly the aircraft, not counting the cost of actually monitoring the behaviour of anything inside it. $\endgroup$ – alephzero Oct 5 '17 at 22:17
  • 1
    $\begingroup$ ... some mandatory engine certification procedures have budgets of the order of 20 or 30 million USD - and double or triple that if the tests are not passed first time and need to be re-done. That sort of money would pay for a lot of CPU chips! But even USD 20m is a fairly small number in a complete "new engine" project with a budget of the order of USD 1bn. $\endgroup$ – alephzero Oct 5 '17 at 22:25
11
$\begingroup$

In general, software isn't manufactured wrong. When the software is created (programmed), defects can be introduced as you described by either faulty implementations or by bad specifications. Faulty implementations are detected by testing the software. Testing takes many forms; unit testing is one of the more basic forms, where individual functions of the underlying programming code is tested to see if it is implemented correctly. This can scale upwards when doing system and integration testing where larger pieces of the software is coupled together to see how it performs as a whole. But simply testing the code at this level doesn't catch everything. Writing a program is rarely about getting it to do what you want it to do, it's mostly about handling all the strange edge-cases and failure scenarios. And this is where most software fail.

To guard against such cases, you can run through audits, simulations, static code analysis and lots of other forms of inspections and testing.

Faulty specifications is a different beast, where you have to rely on documentation. In a perfect world each requirement must be documented to a level describing why the requirement exists, and any input and output that should result from it if applicable. Specifications are developed by multiple people to guard against one person forgetting something, or wrongly interpreting something, but this doesn't catch everything either.

To add another level of protection against software defects, you add multiple instances of the system, and you also have a team create their own version of the systems, preferably on different hardware. You can then divide responsibility of certain subsystems and spread it out among the various computers running the system, adding another level of redundancy as well as to lessen the computational load on each computer, and the risk that any parts of the system interact in unforeseen ways.

The Fast Company had an excellent writeup on the process of writing software for the space shuttle. Although it isn't directly related to neither Airbus or Boeing, it gives an insight into how the process worked and what it resulted in.

$\endgroup$
  • 12
    $\begingroup$ The software for the Space Shuttle was also one of the most expensive software projects ever, in terms of almost every measurable metric. There can be no denying, however, that it did produce quality that most software projects can never hope to even approach (and I say this as a software developer myself). $\endgroup$ – a CVn Oct 4 '17 at 11:54
  • 3
    $\begingroup$ I knew a guy who worked on the shuttle software. I don't think he ever quite recovered from it... $\endgroup$ – Bob Jarvis Oct 5 '17 at 3:23
  • 3
    $\begingroup$ @BobJarvis I'm curious - could you share details? $\endgroup$ – Wumms Oct 5 '17 at 12:37
  • 2
    $\begingroup$ There is a fundamental difference between Space Shuttle and Airbus. Space Shuttle had one implementation of the software and one implementation of the hardware, relying on careful implementation and only using fail-over for in-operation mechanical failures. On the other hand Airbus uses multiple implementations and relies on fail-over for handling both mechanical failures and software and hardware design bugs. $\endgroup$ – Jan Hudec Oct 5 '17 at 16:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.