2
$\begingroup$

I'm searching for a reliable source for a formulization of the QNH adjustment to the barometric altitude. I found this document from weather.gov that gives a formula for the Altimeter Setting, which is QNH if I'm correct. If I replace the formula's values with the appropriate constant names, it becomes this:

$Altimeter Setting (QNH?) = \left ( P - 0.3 \right )\left ( 1 + \left ( \left ( \frac{P_{0}^{\frac{L.R}{g}} L}{T_{0}} \right ) \left ( \frac{H}{\left ( P - 0.3 \right )^{\frac{L.R}{g}}}\right ) \right ) \right )^{\frac{g}{LR}}$

I also know the following formula from the US Standard atmosphere (which is identical to the ISA standard atmosphere up to 50 km):

Pressure Formula

Which can be expresses like this (for troposphere):

$P = P_{0} \left [ \frac{T_{0}}{T} \right ]^{\frac{g.M}{R.L}}$

Symbols and their meanings for extra information:

  • $P$: Pressure at the airport
  • $H$: Altitude at the airport
  • $P_{0}$: Standart pressure (1013.25 $hPa$)
  • $T_{0}$: Standart temperature (288.15 $K$)
  • $L$: Temperature lapse rate
  • $R^{*}$: Universal/ideal gas constant
  • $R$: Characteristic gas constant
  • $g$: Gravitational acceleration of Earth.
  • $M$: Molar mass of dry air

So, my questions are twofold:

  1. Are these formulas actually different reperesentations of the same formula? I tried but I haven't been able to acquire the second formula from the first one.

  2. What is the $-0.3$ for in $(P-0.3)$ in the first formula? Why is it there?

$\endgroup$
3
  • $\begingroup$ If you plug in the figures and get the same answer, or very close, the practical value is confirmed. Unless your aim is to understand the math. I'd try some known rules of thumb like 1 hPa ~ 27 ft ie 27 ft with Std Atmosphere for the rest of the variables should lead to a stn pressure of 1014.25. $\endgroup$
    – skipper44
    Nov 16, 2020 at 14:46
  • $\begingroup$ I've tried it, their results were about 5 hPas apart, which equates to about 40-50 meters of altitude difference. But I am trying to understand the math as well. If I'm gonna use this formula from weather.gov, I need to be able show where it originated from. A reliable source/formula. $\endgroup$ Nov 19, 2020 at 8:53
  • $\begingroup$ Which one seems to be most correct when compared to the ISA -- International Standard Atmosphere - Table ? $\endgroup$
    – skipper44
    Nov 19, 2020 at 14:31

2 Answers 2

2
$\begingroup$

This is a quite old post but I found the same equation on weather.gov a few days ago and I asked myself the same question: where does that formula come from?

It looks like it can indeed be derived from equation (33a). For simplicity, I rewrote it with $H_b=0$ and $\frac{R}{M}=R_s$ as:

$$P = P_0\left(\frac{T_0}{T_0+LH}\right)^{\frac{g}{R_sL}}\tag{1}$$

The only difference with the equation from weather.gov is that $L$ seems to have inverted sign: so in equation (1), one should set $L=-0.0065\frac{K}{m}$ for troposphere. If we want to use $L=+0.0065\frac{K}{m}$ as weather.gov's equation seems to do, we need to rewrite (1) as:

$$P = P_0\left(\frac{T_0-LH}{T_0}\right)^{\frac{g}{R_sL}}=P_0\left(1-\frac{LH}{T_0}\right)^{\frac{g}{R_sL}}\tag{2}$$

If we solve for H (from equation 1) we get:

$$H=\frac{T_0}{L}\left[1-\left(\frac{P}{P_0}\right)^\frac{R_sL}{g}\right]\tag{3}$$

Finding the altimeter setting means finding a specific pressure in the standard atmosphere such that if we position ourselves at the location where that pressure is found and then we go further up by an amount equal to the elevation of the airport, the pressure that we find is the one that we measured at the airport. Or at least, that's what I think altimeters do.

So the elevation of such altimeter setting in the standard atmosphere (which we'll call $H_X$) is:

$$H_X=\frac{T_0}{L}\left[1-\left(\frac{P_X}{P_0}\right)^\frac{R_sL}{g}\right]\tag{4}$$

$P_X$ is indeed what we're looking for, and $H_X$ is the elevation where pressure $P_X$ is found in the standard atmosphere.

The elevation of the airport in the standard atmosphere is instead:

$$H_A=\frac{T_0}{L}\left[1-\left(\frac{P_A}{P_0}\right)^\frac{R_sL}{g}\right]\tag{5}$$

It's worth noticing that while $P_A$ is the actual pressure we measured at the airport, $H_A$ is not the real elevation of the airport.

At this point all we need to do is to solve this equation:

$$h = H_A - H_X\tag{6}$$

where $h$ is the known actual elevation of the airport, and $H_A$ and $H_X$ are equations 4 and 5. I'll spare you the math, but it's easy to prove that (6) yields to:

$$P_X=P_A\left[1+\frac{hL}{T_0}\left(\frac{P_0}{P_A}\right)^\frac{R_sL}{g}\right]^\frac{g}{R_sL}\tag{7}$$

which is the equation you find on weather.gov, a bit rearranged.

Except it's not. I admit I don't know what that $0.3$ is for. It looks like a correction on the reading of the pressure (and 0.3 millibar is really not that much) but I have no idea where it comes from. My guess is that the barometer is usually not located on the ground but a few meters up so reducing its reading by 0.3 millibar is equivalent to subtracting roughly 3 meters and better approximate the pressure on the ground.

$\endgroup$
1
$\begingroup$

Formula 33a (from the US Standard Atmosphere 1976 document you quote) gives you the pressure at a specific altitude in the U.S. Standard Atmosphere, and corresponds closely to airport/station level pressure in met reports (it corresponds exactly when the real atmosphere has the same characteristics as the Standard Atmosphere). An altimeter setting/QNH is essentially a pressure at sea level (and will be 29.92"Hg/1013.2HPa when the real atmosphere matches the Standard Atmosphere, no matter what elevation the airport is).

see the weather.gov definitions here

So the two formulas (33a, and the weather.gov formula you quote) do not give the same thing.

It would be interesting to know why you need to calculate an altimeter setting, as opposed to getting it from met reports. The setting provided in met reports is a value calculated from the measured station level pressure, and this calculation is actually non-trivial.

Beware the frequently-used rule-of-thumb of 27 feet being equal to 1 HPa on an altimeter is only an approximation, and one which only "works" for low elevations/altitudes.

$\endgroup$
2
  • $\begingroup$ Two formulas do not calculate the same values, yes. But they use the same constants and variables. So I thought maybe you could extract one from the other. I don't know the exact mathematical term. If they are different formulas, I need to find where the altimeter setting formula originates from. This is project that I work in, and one that needs proof for solutions. It's interesting that altough non-trivial, this calculation is used worldwide but it's very difficult to find how it is done. $\endgroup$ Nov 26, 2020 at 10:32
  • $\begingroup$ @MelihDurmaz You were asking whether they are "different representations of the same formula", and they are not, even though some of the terms are similar (Note the absence of temperature T at the aerodrome in the equation for QNH). Based on your comment, I suspect you may really be looking for the (non-trivial) formula for reducing station level pressure to sea level to determine QNH. Perhaps try searching those terms, you will find quite a few results. $\endgroup$
    – Ugo
    Nov 30, 2020 at 6:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.