I'm searching for a reliable source for a formulization of the QNH adjustment to the barometric altitude. I found this document from weather.gov that gives a formula for the Altimeter Setting, which is QNH if I'm correct. If I replace the formula's values with the appropriate constant names, it becomes this:

$Altimeter Setting (QNH?) = \left ( P - 0.3 \right )\left ( 1 + \left ( \left ( \frac{P_{0}^{\frac{L.R}{g}} L}{T_{0}} \right ) \left ( \frac{H}{\left ( P - 0.3 \right )^{\frac{L.R}{g}}}\right ) \right ) \right )^{\frac{g}{LR}}$

I also know the following formula from the US Standard atmosphere (which is identical to the ISA standard atmosphere up to 50 km):

Pressure Formula

Which can be expresses like this (for troposphere):

$P = P_{0} \left [ \frac{T_{0}}{T} \right ]^{\frac{g.M}{R.L}}$

Symbols and their meanings for extra information:

  • $P$: Pressure at the airport
  • $H$: Altitude at the airport
  • $P_{0}$: Standart pressure (1013.25 $hPa$)
  • $T_{0}$: Standart temperature (288.15 $K$)
  • $L$: Temperature lapse rate
  • $R^{*}$: Universal/ideal gas constant
  • $R$: Characteristic gas constant
  • $g$: Gravitational acceleration of Earth.
  • $M$: Molar mass of dry air

So, my questions are twofold:

  1. Are these formulas actually different reperesentations of the same formula? I tried but I haven't been able to acquire the second formula from the first one.

  2. What is the $-0.3$ for in $(P-0.3)$ in the first formula? Why is it there?

  • $\begingroup$ If you plug in the figures and get the same answer, or very close, the practical value is confirmed. Unless your aim is to understand the math. I'd try some known rules of thumb like 1 hPa ~ 27 ft ie 27 ft with Std Atmosphere for the rest of the variables should lead to a stn pressure of 1014.25. $\endgroup$
    – skipper44
    Nov 16 '20 at 14:46
  • $\begingroup$ I've tried it, their results were about 5 hPas apart, which equates to about 40-50 meters of altitude difference. But I am trying to understand the math as well. If I'm gonna use this formula from weather.gov, I need to be able show where it originated from. A reliable source/formula. $\endgroup$ Nov 19 '20 at 8:53
  • $\begingroup$ Which one seems to be most correct when compared to the ISA -- International Standard Atmosphere - Table ? $\endgroup$
    – skipper44
    Nov 19 '20 at 14:31

Formula 33a (from the US Standard Atmosphere 1976 document you quote) gives you the pressure at a specific altitude in the U.S. Standard Atmosphere, and corresponds closely to airport/station level pressure in met reports (it corresponds exactly when the real atmosphere has the same characteristics as the Standard Atmosphere). An altimeter setting/QNH is essentially a pressure at sea level (and will be 29.92"Hg/1013.2HPa when the real atmosphere matches the Standard Atmosphere, no matter what elevation the airport is).

see the weather.gov definitions here

So the two formulas (33a, and the weather.gov formula you quote) do not give the same thing.

It would be interesting to know why you need to calculate an altimeter setting, as opposed to getting it from met reports. The setting provided in met reports is a value calculated from the measured station level pressure, and this calculation is actually non-trivial.

Beware the frequently-used rule-of-thumb of 27 feet being equal to 1 HPa on an altimeter is only an approximation, and one which only "works" for low elevations/altitudes.

  • $\begingroup$ Two formulas do not calculate the same values, yes. But they use the same constants and variables. So I thought maybe you could extract one from the other. I don't know the exact mathematical term. If they are different formulas, I need to find where the altimeter setting formula originates from. This is project that I work in, and one that needs proof for solutions. It's interesting that altough non-trivial, this calculation is used worldwide but it's very difficult to find how it is done. $\endgroup$ Nov 26 '20 at 10:32
  • $\begingroup$ @MelihDurmaz You were asking whether they are "different representations of the same formula", and they are not, even though some of the terms are similar (Note the absence of temperature T at the aerodrome in the equation for QNH). Based on your comment, I suspect you may really be looking for the (non-trivial) formula for reducing station level pressure to sea level to determine QNH. Perhaps try searching those terms, you will find quite a few results. $\endgroup$
    – Ugo
    Nov 30 '20 at 6:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.