Relation between altimeter setting (QNH) formula and the barometric formula

Question

I'm searching for a reliable source for a formulization of the QNH adjustment to the barometric altitude. I found this document from weather.gov that gives a formula for the Altimeter Setting, which is QNH if I'm correct. If I replace the formula's values with the appropriate constant names, it becomes this:

$Altimeter Setting (QNH?) = \left ( P - 0.3 \right )\left ( 1 + \left ( \left ( \frac{P_{0}^{\frac{L.R}{g}} L}{T_{0}} \right ) \left ( \frac{H}{\left ( P - 0.3 \right )^{\frac{L.R}{g}}}\right ) \right ) \right )^{\frac{g}{LR}}$

I also know the following formula from the US Standard atmosphere (which is identical to the ISA standard atmosphere up to 50 km):

Which can be expresses like this (for troposphere):

$P = P_{0} \left [ \frac{T_{0}}{T} \right ]^{\frac{g.M}{R.L}}$

Symbols and their meanings for extra information:

• $P$: Pressure at the airport
• $H$: Altitude at the airport
• $P_{0}$: Standart pressure (1013.25 $hPa$)
• $T_{0}$: Standart temperature (288.15 $K$)
• $L$: Temperature lapse rate
• $R^{*}$: Universal/ideal gas constant
• $R$: Characteristic gas constant
• $g$: Gravitational acceleration of Earth.
• $M$: Molar mass of dry air

So, my questions are twofold:

1. Are these formulas actually different reperesentations of the same formula? I tried but I haven't been able to acquire the second formula from the first one.

2. What is the $-0.3$ for in $(P-0.3)$ in the first formula? Why is it there?

Answer 1

This is a quite old post but I found the same equation on weather.gov a few days ago and I asked myself the same question: where does that formula come from?

It looks like it can indeed be derived from equation (33a). For simplicity, I rewrote it with $H_b=0$ and $\frac{R}{M}=R_s$ as:

$$P = P_0\left(\frac{T_0}{T_0+LH}\right)^{\frac{g}{R_sL}}\tag{1}$$

The only difference with the equation from weather.gov is that $L$ seems to have inverted sign: so in equation (1), one should set $L=-0.0065\frac{K}{m}$ for troposphere. If we want to use $L=+0.0065\frac{K}{m}$ as weather.gov's equation seems to do, we need to rewrite (1) as:

$$P = P_0\left(\frac{T_0-LH}{T_0}\right)^{\frac{g}{R_sL}}=P_0\left(1-\frac{LH}{T_0}\right)^{\frac{g}{R_sL}}\tag{2}$$

If we solve for H (from equation 1) we get:

$$H=\frac{T_0}{L}\left[1-\left(\frac{P}{P_0}\right)^\frac{R_sL}{g}\right]\tag{3}$$

Finding the altimeter setting (given the pressure $P_A$ measured at the airport) means finding a specific pressure $P_X$ in the standard atmosphere such that if we position ourselves at the altitude where that pressure $P_X$ is found and then we go up by an amount equal to the elevation of the airport, the pressure that we find is the one that we measured at the airport, i.e. $P_A$

So the elevation of such altimeter setting in the standard atmosphere (which we'll call $H_X$) is:

$$H_X=\frac{T_0}{L}\left[1-\left(\frac{P_X}{P_0}\right)^\frac{R_sL}{g}\right]\tag{4}$$

$P_X$ is indeed what we're looking for, and $H_X$ is the elevation where pressure $P_X$ is found in the standard atmosphere.

The elevation of the airport in the standard atmosphere is instead:

$$H_A=\frac{T_0}{L}\left[1-\left(\frac{P_A}{P_0}\right)^\frac{R_sL}{g}\right]\tag{5}$$

It's worth noticing that while $P_A$ is the actual pressure we measured at the airport, $H_A$ is not the real elevation of the airport.

At this point all we need to do is to solve this equation:

$$h = H_A - H_X\tag{6}$$

where $h$ is the known actual elevation of the airport, and $H_A$ and $H_X$ are equations 4 and 5. I'll spare you the math, but it's easy to prove that (6) yields to:

$$P_X=P_A\left[1+\frac{hL}{T_0}\left(\frac{P_0}{P_A}\right)^\frac{R_sL}{g}\right]^\frac{g}{R_sL}\tag{7}$$

which is the equation you find on weather.gov, a bit rearranged.

Except it's not. I admit I don't know what that $0.3$ is for. It looks like a correction on the reading of the pressure (and 0.3 millibar is really not that much) but I have no idea where it comes from. Reducing the pressure reading by 0.3 millibar is equivalent to adding roughly 3 meters and make the altimeter setting slightly "higher".

As gszlag suggest that's to account for landing gear height.

Answer 2

Altimeter(setting): The 0.3 mb or .01 inHg is deducted from station pressure to account for landing gear height of 10 feet. The derivation of Altimeter and a discussion of the "landing gear offset" as I call it, can be found in the 1951 Smithsonian Meteorolgical Tables, page 269.https://repository.si.edu/handle/10088/23746

Although most might equate Altimeter = QNH, I don't think they are the same. As far as I can tell, QNH has no such offset.

To see this, try inputting an elevation along with the ISA pressure for that elevation into; https://www.weather.gov/epz/wxcalc_altimetersetting and you will see that Altimeter setting comes out to be 1012.95 or 29.91 rather than the expected 1013.25 or 29.92. It comes out short by this rather peculiar elevation offset.

Answer 3

Formula 33a (from the US Standard Atmosphere 1976 document you quote) gives you the pressure at a specific altitude in the U.S. Standard Atmosphere, and corresponds closely to airport/station level pressure in met reports (it corresponds exactly when the real atmosphere has the same characteristics as the Standard Atmosphere). An altimeter setting/QNH is essentially a pressure at sea level (and will be 29.92"Hg/1013.2HPa when the real atmosphere matches the Standard Atmosphere, no matter what elevation the airport is).

see the weather.gov definitions here

So the two formulas (33a, and the weather.gov formula you quote) do not give the same thing.

It would be interesting to know why you need to calculate an altimeter setting, as opposed to getting it from met reports. The setting provided in met reports is a value calculated from the measured station level pressure, and this calculation is actually non-trivial.

Beware the frequently-used rule-of-thumb of 27 feet being equal to 1 HPa on an altimeter is only an approximation, and one which only "works" for low elevations/altitudes.