Motivated by Federico's answer to Can a great circle be drawn between any two points on Earth?

Earth is not a perfect sphere, and it can be argued that the degree of oblateness is negligible. So is the ICAO/aviation adoption of WGS-84 really necessary? Or in other words: when was the need for a truer Earth shape for aeronautical navigation realized?

  • 4
    $\begingroup$ Note: WGS84 is not the true shape of earth, and we have better models. Just US decided that keeping WGS84 for GPS is much better then updating the shape from time to time. And WGS84 is well known and well defined, so it can be converted to other coordinates (this is often not true with old coordinates). PS: "great circle" implies a spherical shape (else one call it just "geodetic"). WGS84 is just the first (and only) wide used (standard for share (and coordinates). $\endgroup$ Apr 14, 2022 at 10:12
  • 2
    $\begingroup$ @GiacomoCatenazzi note that the WGS84 datum is constantly maintained/updated. Last retouch was a few months ago. $\endgroup$
    – Federico
    Apr 14, 2022 at 15:21
  • 1
    $\begingroup$ @Federico: right some parts are updated (mostly about continental shifts: reducing global changes, instead of having fix Greenwich), but no major changes (as in this question: the length of the two semi-axes). It is explicitly described on GPS official documentation. $\endgroup$ Apr 19, 2022 at 7:55

1 Answer 1


The mid-70s was of the earliest mentions of the problem:

In the mid-1970's, during trajectography experiments with the French SAVVAN System (Système Automatique de Vérification en Vol des Aides a la Navigation, i.e. Automatic In-flight Navigation Aids Checking System) positional 'jumps' were noticed when switching between Distance Measurement Equipment (DME) transponders located in different countries. Once more, the errors could only be attributed to incompatibility of the coordinates of ground aids.[1]

This "incompatibility of the coordinates" does not bode well for international cross-border/region navigation. But is it a big jump for a worst-case spherical scenario? It is my understanding that calculating the difference was a mathematical challenge that was solved by computers doing iterative calculations, but the rule of thumb is ±0.3%.[2]

That means when flying at 450 knots ground speed (B737, A320), it will take only around 40 minutes of flying to get an Unable RNP message in RNP-1 airspace, or in other words an uncertainty of 1 NM—that is if the plane's and published datums differ by Earth's oblateness.

There was a time when the lateral separation over the Atlantic was 120 NM, and it wasn't unheard of for 1–2% of the planes to show up after the crossing +40 NM off track.[3] With the continuous rise in air travel, and thus the need for reduced separation, both in oceanic and continental airspace, came the need for more accurate navigation. It's unthinkable to have the ATC nudge every single plane back on track by giving minute vectors when the airspaces are running near capacity.

Notable mentions:

  • GPS If GPS, augmented or not, didn't use a truer shape, applications that need geometric altitude wouldn't be feasible, e.g. LPV approaches and EGPWS. Related: Does altimeter setting affect the vertical guidance in a LPV approach?

  • INS Inertial navigation already must account for a spheroid shape for the purposes of finding the local vertical (example Sperry patent), without which, a built-in error of ~12 NM would be unavoidable.[4]

    • Accelerometers, like bubble levels, can measure either acceleration, or level, but not both at the same time. So the basic principle is fixing the accelerometers in space, and accounting for the gravitation (via e.g. torque motors based on Earth's local radius, or computationally);[5] any simple erecting mechanism will be subject to acceleration (affecting the measurements), like a glass of water for attitude indication.
  • RNAV Even pre-WGS-84 adoption, in RNAV systems the patents already show the engineers accounting for the spheroid shape, as in this Honeywell patent from the 80s.

  • ATC For air traffic control/management, it was in the 80s with the more stringent specifications that more accurate surveillance was needed—the issue also affects the projection of moving targets.[6]

1. European Organization for the Safety of Air Navigation (EUROCONTROL) and Institute of Geodesy and Navigation (IfEN). "WGS 84 Implementation Manual." (1998). (PDF)
2. Geyer, Michael. Earth-referenced aircraft navigation and surveillance analysis. No. DOT-VNTSC-FAA-16-12. John A. Volpe National Transportation Systems Center (US), 2016. (PDF)
3. White, A. "Air Traffic Control Separation Minima and Navigational Capability." The Journal of Navigation 24.4 (1971): 443-456.
4. Pitman, George R., and J. D. Trimmer. "Inertial guidance." American Journal of Physics 30.12 (1962): 937-937.
5. King, A. D. "Inertial navigation-forty years of evolution." GEC review 13.3 (1998): 140-149. (PDF)
6. Mulholland, Robert G. On the Application of Stereographic Projection to the Representation of Moving Targets in Air Traffic Control Systems. Federal Aviation Administration Technical Center, 1985. (PDF)

  • $\begingroup$ Good question and answer! Just a comment and question on your first two notable mentions: First, regardless of the accuracy of your non-spherical earth model, without augmentation from ground based stations you can't obtain the vertical accuracy required to shoot an LPV approach. Second, why would an INS need a spheroid shape to find local vertical? A vacuum driven gyro can find local vertical. (Heck, a rock on a string can find it too...) In fact, couldn't an INS theoretically function on a flat earth? $\endgroup$ Apr 14, 2022 at 0:54
  • $\begingroup$ "Errors could only be attributed to incompatibility of the coordinates of ground aids" - this doesn't necessarily relate to oblateness or 'truer' shape of the Earth. Incompatibility could relate to the differences in the reference meridian or other parameters. Also, prior to adoption of WGS, systems were mostly locally-optimised; for example, Pulkovo-42 was more accurate for Eastern Europe than the much later WGS 60 (both use the same flattening), and much more accurate than the contemporary international standard. $\endgroup$
    – Zeus
    Apr 14, 2022 at 1:14
  • 1
    $\begingroup$ @MichaelHall An already aligned INS could function on a flat Earth, but how would the alignment work? INS alignment senses the Earth's rotation and can accurately determine your orientation from that alone. But that requires knowing about the non-spherical shape to determine local vertical. The erection system of a vacuum gyro uses gravity for local vertical, but the INS doesn't. Even after alignment, the non-spherical shape is important to keep local vertical perpendicular to the Earth's surface when moving to a different location. $\endgroup$
    – Bianfable
    Apr 14, 2022 at 6:05
  • $\begingroup$ @Zeus: Very informative insight as usual. I've added a sentence to get my point across clearer. $\endgroup$
    – user14897
    Apr 14, 2022 at 8:25
  • $\begingroup$ @MichaelHall: Thanks for the compliment. I added remarks to both points. $\endgroup$
    – user14897
    Apr 14, 2022 at 12:58

You must log in to answer this question.