This is a fascinating question and its answer would arguably be better suited to a rather longer article than a Stack Overflow post: you're effectively asking (1) what is the history of the development of the ICAO standard atmosphere model, and (2) what version of it therefore was used -- and therefore (3) what is different now compared to then to explain the discrepancy.
Briefly, the answers to these are (1) – very complicated (2) – complicated and (3) an improved understanding of the tropopause and a better quantification of features of the atmosphere such the dry adiabatic lapse rate (which I think wouldn't have been used in the 19th century, but would have been in the 20th).
(Just as a reminder, the difference in the tropopause matters, because, in modern notation, atmospheric pressure varies with height as $\frac{dP}{dh}=-\rho g$ and the ideal gas law in molar form gives us $P=\rho R T(h)$ -- so you need to know solve both simultaneously in order to work out height from pressure).
It transpires that it is important to keep in mind that the flights you mention occurred in 1838 in Britain (by Green and Rush – who were British) and in 1932 in Switzerland (by Cosyns and Picard – who were Belgian and US citizens respectively).
Probably the best reference to the early history I can easily find is this review from 1977 [1] to put the then-new 1976 Standard Atmosphere in context. Quoting from it extensively to answer the developments in the 20th century first (I shall return later to the 19th):
[...] Aneroid barometers were calibrated [...] in accordance with an isothermal standard atmosphere. A British standard
atmosphere of this era [Airy, 1867] provided a table of aneroid
pressures as a function of height from 0 to 3657.6 m (12,000
feet) computed on the assumptions of a sea level pressure of
31.0 in. (787.4 mm) of mercury and a constant mean temperature of 50 ºF (10ºC)
[...] This model was used as the basis of barometric altimetry in the US until after World War 1. During the early part of the twentieth century the French had a different standard atmosphere for height determination, based upon a nonlinear temperature-height function proposed by Radau [1901]. According to Grimault [1920] the French aeronautical community until 1920 depended upon this 'Law
of Radau' as the temperature-height function in a standard
atmosphere used for normalizing measurements of aircraft
performance.
This didn't quite work, however, as the $P, T$ relationship deviated substantially from reality as measured at altitudes above 5 km, and after 1920 atmospheric sounding (balloon) measurements were taken up to heights of around 14 km. This discrepancy, between physics and reality, was highlighted by an Italian military engineer, Gamba, when he published a graph of the ten-year averaged data of temperature vs height in the range of $(0, 20)$ km in the Pavie region of Italy. These measurements lead to a piecewise linear adjustment to the model:
Using Gamba's curve as a basis for comparison, Toussaint [1919] noted the limitations of the Radau temperature-height expression and showed the close agreement between Gamba's observations and a two-segment linear temperature-height function consisting of a constant negative gradient, -0.0065 ºC/m from +15ºC at sea level to -56.6 ºC at 11,000-m altitude, and a zero gradient from 11,000- to 20,000-m altitude.
It appears that this model had some apparent success and was closer to reality -- at least, as far as early aviators and metrologists could tell. Naturally, the US decided to do its own thing.
Toussaint computed tables of atmospheric pressure, temperature, and density as a function of altitude in increments of 500 m from 0 to 11,000 km and in increments of 1000 m from 10,000 to 20,000 m. This proposal led the French [Grimault, 1920], in April 1920, to adopt the Toussaint temperature model as the basis of a new aeronautical standard atmosphere. Grimault, noting that the available meteorological data (primarily for western Europe) generally supported the Toussaint model, emphasized the advantages to be gained if cooperating European countries would each adopt the same standard. Italy and England soon followed this advice [British Aeronautical Research Committee, 1922], while the United States,apparently intending to do likewise,adopted only a portion of that model, a situation which led to confusion shortly thereafter and to some disagreement many years later.
Why? Well, in 1921 the US executive committee of the National Advisory Committee for Aeronautics (NACA) met, and they adopted for performance testing Toussaint's formula linking temperature, pressure and height.
This statement, with no height boundaries and no mention of an
isothermal region,appears in Gregg's document,which limited the listing of computed atmospheric properties to heights between 0 and 10,000 m while showing observed properties for a much greater height range. This document implicitly limited the maximum height of the 1922 U.S.Standard Atmosphere to about 10,000 m (33,000 feet).
What happens next is fairly typical: Europe develop Toussaint's formulae and model in one direction, and the US in another, and a discrepancy arises. The details of this happen to correspond exactly in time to the period you ask about:
The growing requirements for standard atmosphere tables by the aeronautical community as well as by artillerists led Diem [1925] to expand the U.S. Standard Atmosphere. This expansion included the following points: (1) increasing the height resolution; (2) adding listings of ratios of the several properties to their sea level values; and (3) extending the altitude range from a maximum of 10,000 m up to 20,000 m (but in a manner not consistent with the Toussaint
model in the ICAN standard) [emphasis mine]
Diehl, like Gregg, appears to have been unacquainted with
the complete Toussaint model as well as with the actions of
ICAN, for while he properly extended the range of the constant lapse rate of temperature (-0.0065ºC/m used by Gregg) to heights above 10,000 m, he unfortunately extended it only to a height of 10,769 m (see $G D_1$ in Figure 1), where the computed temperature becomes -55ºC (in accordance with Gregg's mean of observed temperatures for 45ºN in the United States). Then from 10,769 m to 20,000 m altitude, Diehl introduced an isothermal layer at -55ºC (see $D_1 D_2$ in Figure 1) instead of the one previously adopted at -56.5ºC in the Toussaint-ICAN standard ($T_1 T_2$ in Figure 1). The slight difference in the height and temperature of the tropopause (231 m and 1.5ºC, respectively) later led to international aeronautical confusion which was not resolved until after World War II. [emphasis mine]
In the interim it was decided that in the United States, in accordance with the actions of ICAN, altimetry would no longer be based upon an atmosphere isothermal at 10ºC [National Bureau of Standards, 1920] but rather would be based on the same standard used for comparison of aircraft performance. The metric altimetry tables of ICAN, however, were neither in agreement with the U.S. Standard Atmosphere [DieM, 1925] at heights above 10,769 m nor convenient in the United States, where measurements were made primarily in English units. Furthermore, Diehl's English tables of pressure versus height
were not conveniently formulated for altimeter calibration. Consequently, Brombacher [1925, 1935], of the U.S. Bureau of Standards, prepared tables of altitude as a function of pressure as well as other tables particularly applicable to the calibration
of altimeters.
I therefore propose that it is likely that some variant of the Toussaint model, either in the convenient pressure or temperature table -- due to Brombacher in 1925 (available online [2]), or the slightly different European version codified by the predecessor to ICAO, ICAN [3] -- to have been that used for the 1933 flight. Exactly which version probably needs detailed historical investigation to determine: it depends on the detail of their equipment's manufacturer and I can't find a good amount of info on that online, but I think it's going to be the US one (Brombacher 1925) as (i) Picard was a US citizen (ii) the use of that model results in a larger discrepancy and possibly a larger height, and (iii) most certainly it would have been convenient to use at the time as the actual publication provides a look-up table of height vs pressure. It's quite clear, however, that the difference between our "modern" ICAO atmosphere and measurements at that time largely coming from the understanding of the tropopause (set in 1976 as being 11 km geopotential altitude and at -56.6 ºC – very close to Toussaint's model) – I haven't (yet) done a detailed calculation to see if it explains all of it mathematically, but one could. Edit - it transpires it's a bit hard to recreate Toussaint's original publication because I don't have access to it, and I wouldn't understand physics in French if I did (physics in english is fine ;-) ). However, as a rough demonstration of the size of the effect this gives, here's a plot of a very hacky numerical solution to the two different atmospheres as a function of height, up to the stratosphere – although the tropopause is at 11 km, they're quite different before that:
These discrepancies in atmospheric models were slowly, slowly ironed out over much of the next 30 to 50 years (spurred on, I suspect, by increased aviation co-operation during WW2), and the final standard atmospheric models we have today contain detailed information well outside of the Kármán line, as is required by spaceflight operations.
The situation for the Victorian aero-naught is rather different, and I have to say, harder to pin down. At present, I can't easily access the original works of Mr Airy, The Astronomer Royal on determination of heights from barometer readings, Proc. Roy. Meteorol. Soc., 3, 407-408, 1867, which I suspect contains a pressure-height curve in detail (not least as Fig 1 above indicates that it is isothermal), but I am aware that the knowledge of physics behind this is far older: the relationships between pressure, volume and temperature are largely 17th century in discovery and 19th century in detail (the ideal gas law was written down in one unified form in 1834 by Claperyon). M. De Luc writes in the Philosophical Transactions of the Royal Society in 1774 a detailed account of solving this problem with the most wonderful title [4]:
M. De Luc's Rule for Measuring Heights by the Barometer, Reduced to the English Measure of Length, and Adapted to Fahrenheit's Thermometer, and Other Scales of Heat, and Reduced to a More Convenient Expression. By the Astronomer Royal.
To cut quite a lot of the 13 pages of Georgian verbage and vital experimental detail and notes such as the differences between French and English feet (!!!), he espouses, entirely in the third person:
[...he] thus corrected the height of both of his barometers (that at the bottom, and that at the top of the hill) for the particular degree of heat, indicated by a thermometer attached to the barometer [... and] calling the two altitudes of the barometer $B$ and $b$, and putting $\log B$ for the logarithm of $B$ [...] and calling the heights of this [carefully calibrated] thermometer at the two stations be called $G$ and $I$ [...]
[...] the formula that will give the height of the upper station above the lower one, in English fathoms will be $$\log B - \log b \mp d \times I + \frac{G+I}{1000}$$.
I think it is possible to probable that approaches of this sort were used by Green and Rush; an 1863 publication (again in Phil Trans) entitled "On a Simple Formula and Practical Rule for calculating Heights barometrically without Logarithms" [5] does much the same thing using a more complicated linear approximation that involves a cosine of latitude, lookup tables and, frankly, is equally awful (to the modern eye). All of these were interested in assessing elevation rather than altitude per se -- in other words, how tall is that mountain, rather than how high is my balloon, but I am somewhat convinced that the basic physics would have been understood at the time, because the earliest balloon flights used a barometer as an altimeter (the first recorded use is, I think, in the Charliére, the successor to the Montgolfiére, in December rather than September 1783 [6]), presumably with an isothermal atmosphere. However, I can't answer this more concretely and it is difficult to know exactly what model would have been used at the time – if evidence emerges, I reserve the right to edit this in the future.
An obligatory, relevant XKCD to end with:
References
[1] Minzner, R. A. (1977). The 1976 Standard Atmosphere and its relationship to earlier standards. Reviews of Geophysics, 15(3), 375. https://dx.doi.org/10.1029/rg015i003p00375
[2] Brombacher, W. G. (1925). Tables for calibrating altimeters and computing altitudes based on the standard atmosphere. https://ntrs.nasa.gov/citations/19930091313
[3] International Commission for Air Navigation, Official Bulletin,vol. 9, p. 27, resolut. 226, Montreal, 1922. https://books.google.dk/books/about/Bulletin_Officiel.html?id=CAngAAAAMAAJ&redir_esc=y
[4] De Luc, M. (1774). M. De Luc's Rule for Measuring Heights by the Barometer, Reduced to the English Measure of Length, and Adapted to Fahrenheit's Thermometer, and Other Scales of Heat, and Reduced to a More Convenient Expression. By the Astronomer Royal. https://archive.org/details/philtrans01544508/page/n11/mode/2up
[5] Ellis, A J. (1863). On a Simple Formula and Practical Rule for calculating Heights barometrically without Logarithms. https://dx.doi.org/10.1098/rspl.1862.0107
[6] Gibbs-Smith, C. H. (2010). Notes on the History of Ballooning. https://www.cambridge.org/core/journals/journal-of-navigation/article/abs/inotes-on-the-history-of-ballooning/5FD910C7F47D61FE80FD16D4CC7BC9F8
