Journal of Geophysical Research: Atmospheres

An evaluation of the expression of the atmospheric refractivity for GPS signals



[1] An expression is derived from first principles for the refractivity of air at L band frequencies, which includes GPS, as well as other GNSS satellite radionavigation signals. Under conditions of pressure, temperature, and moisture content found in the Earth's atmosphere, the expression has an average relative error of approximately 0.01%. This level of accuracy is required to guarantee that the expression does not introduce bias, when it is used within the context of Numerical Weather Prediction (NWP) applications. The thermodynamic dependences of the air's refractivity N are revisited, and the possible sources of uncertainty are analyzed. A first principles microphysical model is constructed, which relates the refractivity at L band frequencies with several measurable properties of matter. The experimental values that are critical for this purpose are already available in the literature and are of high accuracy. Based on this model, a simple expression suitable for atmospheric and weather applications is proposed: N ≡ (n − 1) · 106 = N0 · (1 + equation imageN0) where N0 = (222.682 + 0.069 · τ) · ρd + (6701.605 + 6385.886 · τ) · ρw with ρd and ρw as the densities of dry air and water vapor in the air (kg/m3), τ = 273.15/T − 1, and T as the absolute temperature in K. The dependence of the coefficients in the expression with respect to the input physical parameters is analyzed. Given the error of the experimental parameters, it is concluded that the proposed expression improves the accuracy to meet the needs of NWP applications.

1. Introduction

[2] GPS radio occultation (GPSRO) data are being used operationally at a number of numerical weather prediction centers [Healy and Thépaut, 2006; Cucurull et al., 2007; Rennie, 2008; Aparicio and Deblonde, 2008; Poli et al., 2009]. One of the reasons for the success of this observing technology is its good absolute calibration. For upper air measurements, only radiosondes have a traceability to fundamental standards of comparable quality, among observational technologies available today.

[3] Numerical Weather Prediction (NWP) applications require a strict level of compatibility in the relative calibration between different measuring technologies and weather models. For many meteorological measurements, especially for radiances, it is not possible to have a complete understanding of all physical processes involved in the procedures of measurement, calibration of instrument and data assimilation, to the level of accuracy required to consider the data as absolutely calibrated. The imperfect understanding of the traceability chain is termed bias, and an ad hoc procedure of bias correction is applied that brings data to a sufficient level of coherence with other data sources, and with the physical processes described in NWP models. It has been a common practice to use radiosondes as the fundamental measurement that is assumed to be physically well understood and meteorologically traceable.

[4] Since raw GPSRO measurements are very accurate and their calibration is traceable, it would be of little efficiency to downgrade them within the context of NWP by accepting that there is some bias of unknown source, which has to be independently determined. If GPSRO data are used instead as an additional absolute reference, together with radiosondes, it is necessary that radiosondes and GPSRO, whose traceability to fundamental standards is entirely independent, are mutually coherent up to the level of accuracy required for compatibility, and therefore that all physical processes involved in both technologies are indeed understood up to that level of accuracy. All physical processes relevant for the interpretation of these data have then to be accounted for.

[5] It has been noticed [Aparicio et al., 2008, 2009] the high sensitivity of NWP systems to details of the calibration, and in particular to the coherence between GPSRO and radiosonde measurements. In those studies it was found that at least some standard assumptions for the assimilation of GPSRO data were not sufficiently accurate to guarantee that coherence. Effects whose size is on the order of 0.05% are nonnegligible. The nonideal behavior of the air was in particular identified [Aparicio et al., 2009] as one of the physical processes whose impact is not sufficiently small to be ignored, if GPSRO data are to be considered free of bias within the context of NWP. Similarly, Healy [2009], Cucurull [2010] and Healy [2011] found that the relationship between refractivity and other thermodynamic variables is not currently known with sufficient accuracy, and that its margin of error is sufficient to lead to detectable differences in the forecast performance of NWP systems. It is thus clear that although GPSRO data have proven to be useful in data assimilation, better accuracy in the thermodynamical relationship between refractivity, pressure, temperature, and moisture is required to fully exploit the good traceability of the raw data. The requirements of NWP are sufficiently demanding to justify a careful revision of all steps, in order to safely consider these data as unbiased. The mentioned implementations and tests across several operational NWP centers differ in their details by amounts on the order of 0.1%, and are otherwise successful in providing a very positive impact from the assimilation of GPSRO data.

[6] The aim of this study is to review all physical processes involved, in order to propose a refractivity model whose absolute accuracy exceeds 0.1%, which is assumed to have been already realized and implemented, and progresses as much as possible toward 0.01%. It is an additional aim to identify the physical parameters whose imperfect knowledge currently imposes a limit to the accuracy of this model, and where further experimental results would be necessary in order to improve the accuracy of the model presented here.

[7] In section 2, the ensemble of physical processes involved in the refractivity of the atmosphere is described. Beyond the elements that are nonnegligible at the mentioned target accuracy, the ensemble includes several elements which are judged sufficiently small to be negligible. They are included to show that these are indeed negligible, and to allow the estimation of the level of required accuracy where some elements, whose impact is here marginal, would begin to be relevant. For an average parcel of air at sea level, the refractivity is approximately 300–400 N units (the refractivity is adimensional, and defined as N = 106 · (n − 1), where n is the refraction index). The target of 0.01% allows a total error of 0.03 N units at surface conditions. In order to allow this, all processes that imply effects at an approximate level of 0.01 N units, or above, are explored further. We only neglect processes that at surface conditions have an impact on refractivity obviously below 0.01 N units.

[8] A physical model that relates microscopic molecular properties and macroscopic constitutive properties of air is then constructed in section 3. The model allows the evaluation of the atmospheric refractivity from more fundamental parameters, and also the estimation of the sensitivity with respect to these parameters, within the context of an atmospheric environment. This sensitivity analysis is applied to the estimation of the error of the proposed model, and points to those fundamental parameters whose accuracy would have to be improved if higher accuracy of the final model is needed.

[9] In section 4, the model is simplified to a practical expression for use within the current context of NWP, and suitable for similar applications in atmospheric physics. The practical expression is shown to represent the underlying physical model with very high accuracy.

[10] In sections 5 and 6, a review is made of several available expressions, compared against the expression proposed in section 4.

2. First Principles Model

[11] This section details the model that describes the relationship between microscopic molecular properties and macroscopic refractive properties. Let us assume that the air has a number of macroscopic properties: pressure p, absolute temperature T, mass density ρ = ρd + ρw that includes dry air ρd and water vapor ρw, index of refraction n, refractivity N ≡ 106 (n − 1), electric susceptibility χe and relative dielectric constant εr ≡ 1 + χe, magnetic susceptibility χm and relative magnetic permeability μr ≡ 1 + χm, and is constituted by a mixture of substances i, (in this case nitrogen, oxygen, water vapor, etc) with number concentrations ni. In microscopic terms, the molecules of each participating substance have properties such as average polarizability αi, permanent electric dipole μi, mass mi, molecular magnetic susceptibility χi, and permanent magnetic dipole νi.

[12] The refractive index of a propagating material for electromagnetic (hereafter EM) waves is given by Maxwell's equation [Born and Wolf, 1999],

equation image

The material properties εr and μr depend in general on the frequency of the propagating signal. This study focuses on the response of air to GPS signals, or in general to EM waves of a frequency in the range of 1–2 GHz, in the L band. At these frequencies, absorption and dispersion by the air are very small, and will be neglected.

[13] A detailed and quantitative treatment of EM absorption and dispersion by air at radio frequencies is described by the MPM-93 model [Liebe et al., 1993]. It provides an evaluation of both absorption and dispersion. We have verified their size for the frequencies of interest, in the L band, and found that the assumption that both are negligible is appropriate. The closest resonances are around 23 GHz (water vapor) and 50–60 GHz (molecular oxygen). All other relevant resonances lie at higher frequencies. Our goal here is to describe the behavior of air at frequencies substantially lower than that of any resonance (i.e., the static limit). Although the region of 1–2 GHz is safely away from distant resonance frequencies, it is still important to have the existence of those in mind, since most available measurements of material EM response have not been obtained at exactly the GPS frequencies, and some measurements have been performed close to resonance frequencies.

[14] It is assumed in this work that the refraction index n is real (no absorption), and that any dependence with respect to frequency is negligible (no dispersion). The approach has been to select those measurements of EM properties that have been obtained at sufficiently low frequencies, even if not exactly at GPS frequencies, and far from resonance frequencies, so that they are representative of the same frequency regime as the L band. Experimental measurements for which the MPM-93 model suggests that there is a relevant difference (significant dispersion) between the L band and the test frequency are avoided. This most notably affects measurements of O2, which has low-frequency resonances. According to the MPM-93 model, measurements of O2 EM response are representative of the behavior at 1–2 GHz only at test frequencies below 10 GHz, within the margin of error considered acceptable for this study. Some accurate measurements [e.g., Newell and Baird, 1965] of O2 EM response have been avoided due to this reason. For all other components of dry air, MPM-93 suggests that measurements obtained at up to 100 GHz represent accurately the behavior at 1–2 GHz.

2.1. Electric Permittivity

[15] The electric permittivity of the propagating material is related to microscopic properties through the Lorentz-Lorenz relationship [Born and Wolf, 1999],

equation image

where kB is the Boltzmann constant and f is a function of the dielectric constant εr, that expresses the interaction of a dipole with the surrounding particles. The quantity f depends on the permittivity of the material, has a low density limit of 1, and for air remains always very close to 1.

[16] We term the right side in equation (2) as δε/3, which is small for air. The Taylor expansion for εr yields

equation image

[17] For low densities, equation (2) can be reduced to the first term in this expansion, and with f = 1, obtaining the linear approximation to the electric susceptibility, εr0,

equation image

Equation (2) is only weakly nonlinear for a substance like air at atmospheric conditions, and the linear solution (4) is already quite good. In this linear approximation, the electric susceptibility εr0 − 1 is proportional to the number density of each kind of molecule. Also, for each substance, contributions stemming from the molecular polarizability and molecular dipole are independent. Within this low-density limit, the contribution of each substance and microphysical process is therefore separable. This linear model is not sufficiently accurate for the purpose of this study. However, it is sufficient to identify the input parameters that are critical, and the approximate contribution per substance and per microphysical property to the total refractivity.

[18] Regarding the nonlinear behavior, equation (2) converges very quickly under iteration,

equation image

2.2. Magnetic Permeability

[19] For most substances, and in particular for the components of air, the electric response to an EM wave is much larger than the magnetic response. The relative electrical permittivity of air at mean sea level (MSL) pressure is approximately εr ≈ 1 + 6 · 10−4, whereas the relative magnetic permeability is approximately μr ≈ 1 + 4 · 10−7. All air substances, both in dry air and water, have a small magnetic susceptibility. The substance with the largest magnetic susceptibility is molecular oxygen, which has a nonnegligible paramagnetic response due to a permanent molecular magnetic dipole, and largely dominates the magnetic susceptibility of air as a mixture, either dry or moist. As will be detailed below, approximately 0.2–0.3 N units of refractivity can be attributed to the magnetic susceptibility of oxygen in the low troposphere. Nearly all other components of air are diamagnetic (negative magnetic susceptibility), but their response is much weaker than that of oxygen. An equation similar to that of Lorentz-Lorenz can be applied. Since the magnetic response is very small, and is in fact barely above the threshold of tolerable error for this study, the relationship can be safely linearized,

equation image

The molecular magnetizabilities χi are generally very small and negative, most substances being diamagnetic. The molecular permanent magnetic dipoles νi are only significant for O2, which dominates the permeability of the mixture.

2.3. Functional Form

[20] Following the low-density expansions already mentioned, with most of the refractivity caused by the dielectric permittivity, Maxwell's equation for the refractivity N = 106(n − 1) = 106(equation image − 1) could be expressed as

equation image

We keep the electric susceptibility to second order, but the magnetic susceptibility, much smaller, only to first. This expansion allows the definition of a first approximation to the refractivity,

equation image

This quantity is also linear in the number densities ni. The asymptotic (low density) dependence on the molecular polarizabilities and magnetic susceptibilities appears as a constant times ni. The asymptotic dependence on either the electric or magnetic molecular moments appears as a constant times ni/T. For air, the only relevant moments are the electric dipole of water, and the substantially small permanent magnetic moment of O2. For this reason, we can expect that an ansatz function such as

equation image

will be a good fit in the low-density limit. The quantities q1, q2, q3 and q4 are constants that need to be determined. In the absence of nonlinearities, these quantities would represent the polarizability of dry air (q1), the magnetic moment of O2 (q2), the polarizability of water vapor (q3), and the electric dipole of water vapor (q4). Most existing expressions in the literature follow similar functional forms, but expressed in terms of pressure,

equation image

although the dependence represented by the parameter k4 is neglected in all expressions commented below.

[21] Expressions such as (9) or (10), as written, have the theoretical value of underscoring the relationship between microscopic properties (molecular polarizability, permanent moments), and the macroscopic refraction index. From a practical point of view, however, the coefficients are determined from a fit to either experimental or model-generated values. Within this fit, some of the parameters are strongly correlated. This happens with the values of k2 and k3 [e.g., Rüeger, 2002]. It will happen similarly with the pairs q1 and q2, and with q3 and q4. We therefore choose an expression for the low-density behavior with the same degrees of freedom as (9), but with less correlated parameters,

equation image

with τ = 273.15/T − 1. The densities are linked to variables commonly used in meteorology, such as pressure P and specific water vapor content q by the equation of state P = ρRTZ, where ρ = ρd + ρw is the total density, and Z is the compressibility factor, and where q = ρw/ρ. We use the compressibility factor recommended by Picard et al. [2008].

[22] Regardless of the linear expression used, which theoretically represents the low-density limit of the refractivity, nonlinear dependencies are not entirely negligible with the target of accuracy of this study, and is illustrated in Figure 1. We then choose an expression for the refractivity that reduces to equation (11) in the low-density limit, but where we impose that its quadratic dependence on density follows equation (7). In this way, nonlinearities can be represented without the introduction of new degrees of freedom to the fit. The result is the following:

equation image

where N0 is assumed to be accurately representable by an expression such as (9), or its less correlated equivalent (11). This is the functional form, or ansatz, that will be fitted in this work in order to reduce the entire model to a simple expression. We note that we are imposing that the nonlinear dependence on density follows equation (7), where we only considered worth retaining the nonlinear dependence of the electric susceptibility, but not the magnetic one. Although the magnetic susceptibility is not neglected, we neglect its nonlinearity, and therefore its impact in the nonlinear term of the refractivity with respect to the density.

Figure 1.

Excess refractivity of the full refractivity model presented in this study (equations (2) and (7)) with respect to the linear approximation (equations (4) and (8)). The excess is evaluated over a wide range of realistic atmospheric conditions (see section 4).

[23] It is also interesting to note that this functional form, quasi-linear in the densities of the different substances, deviates from linearity due the expression of the electric permittivity (2), and to the presence of a square root in Maxwell's equation (1). The former represents the enhancement of the microscopic polarization due to the polarization of the environment of each molecule, and is a positive nonlinearity. The latter is negative and it partially compensates the first. The net effect is that refractivity grows slightly faster than linearly in density, which is illustrated in Figure 1.

3. Relevant Processes and Parameters

[24] In section 2, a general model for the electric and magnetic susceptibilities of air was presented. It contains numerous parameters concerning microscopic properties of the molecules contained in the air. The different parameters required are reviewed in sections 3.13.7.

3.1. Composition of the Air

[25] In an atmospheric environment, air can be well approximated as a mixture of two components: dry air and water vapor. Dry air is a mixture of very uniform composition below 80 km above MSL. For meteorological applications, only some of the minor traces show local variations, such as methane, ozone and nitrogen oxide. As shown below, these traces do not contribute significantly to the refractivity of dry air, and are included here only to verify that their contribution is indeed negligible. For all present purposes, dry air has thus a homogeneous composition. In order to verify any dependence with respect to small traces, a total of 12 substances are included in the dry air fraction. Only a few have a significant contribution to the dry air refractivity at the level of accuracy requested in this study. Relative quantities of the different molecules are listed in Table 1. The compositions are by number, with respect to the dry fraction of the air. Most mole fractions are taken from NOAA et al. [1976], with some minor adjustments, such as an updated Ar content [Picard et al., 2008], and an updated approximate value of CO2. All these molecules are nonpolar. Approximate values of some variable minor traces (O3 and N2O) not included in those compilations are included here. They have been added since these are polar molecules, in order to explore the polar refractivity of dry air. The tabulated fractions of dry air are renormalized by the numerical model described here so that, within the model, their sum is exactly one. The average mass are obtained from the isotopic compositions described by Böhlke et al. [2005].

Table 1. Dry Air Substances Considered in This Study and Their Assumed Fractions and Propertiesa
SubstanceMole FractionWeightα0αt/10−3μiχi
  • a

    A constant approximate concentration is assumed for all, including traces that are known to show important local variations, such as O3, N2O and CH4. Molecular polarizabilities α are in units of 10−24 cm3. We note that in SI units of cm2/V, α (cm2/V) = 4πε010−6α (cm3)). Molecular permanent electric dipole moments μi are in Debye units D (D = 3.33564 · 10−30C · m). Magnetizabilities χi are in units of 10−12/mol. Oxygen's magnetizability is temperature dependent (see main text). The value in the table is for T = 273.15 K.


3.2. Electric Susceptibility

[26] For the electric susceptibility, we have chosen the experimental measurements of the dielectric properties of N2, O2, Ar and CO2 presented by Schmidt and Moldover [2003]. This covers all species for which accurate values are critical for this study. Additional studies with O2, including measurements of its magnetic susceptibility, are presented by May et al. [2008]. For all other species in dry air, we use the values from Lide [2001]. For the compilation in Table 1, all data were transformed to the same units of molecular polarizability.

[27] In the findings of Schmidt and Moldover [2003], the fit to experimental data indicates a temperature dependence of the polarizability α, for N2 and O2. This dependence is small, but was nevertheless included in this model. The dependence follows the form

equation image

We take αt according to Schmidt and Moldover [2003] for N2 and O2, and zero for all other species. These authors also considered density dependence of the polarizability. This variation, however, is small even at the highest densities tested by their experiments. At atmospheric densities, the impact of this density dependence is sufficiently small to be neglected. It was therefore not included in this model.

3.3. Magnetic Susceptibility

[28] Approximate values of the magnetic susceptibility of the substances involved are provided by Lide [2001]. Most of these substances have a negative magnetic susceptibility, but very small and negligible. The only substance for which this is not negligible, although still small, is O2, with a positive susceptibility derived from its permanent magnetic moment. Lide [2001] cites a constant magnetic molar susceptibility for all relevant substances. This is appropriate to describe the small diamagnetic behavior of most air molecules, although it does not accurately describe oxygen. The value provided for O2 is image = 3449, in units of 10−12/mol. The total magnetic susceptibility χiT of a substance i, if it contains a magnetic dipole, can be described as in equation (6), that is

equation image

[29] An accurate model for O2 is provided by May et al. [2008], which contains a small diamagnetic contribution χi, whose size is similar to that of other molecules in dry air, and a much larger paramagnetic contribution, derived from the alignment of its magnetic moment to the test field. Thermal motion prevents this alignment, and this term is therefore temperature-dependent, stronger at lower temperatures. For this study, we have chosen May et al. [2008] for O2, and Lide [2001] for all other substances. When reduced to the same units, the temperature-dependent expression by May et al. [2008], translates to image = −10 + 3677 · 273.15/T. Table 1 presents magnetizabilities of the different substances. The value shown for O2 corresponds to T = 273.15 K.

3.4. Isotopic Composition of Dry Air

[30] The substances included are not isotopically pure. However, they are all dominated by an isotope whose abundance is much larger than that of the other isotopes. For instance [Böhlke et al., 2005], molecular nitrogen is mostly 14N2 (99.27%) versus 14N15N (0.73%). Oxygen is mostly 16O2 (99.52%), most of the rest being 16O18O (0.40%). The presence of isotopes can have implications in two different ways. First, through the mean molecular mass of each species, and consequently in the relationship between pressure and density. This is small, but still significant at the level of accuracy that is targeted in this study. This can be easily accounted using the appropriately averaged molecular mass. Secondly, the optical properties of a molecule can change slightly between the different isotopologues of the same species. Different isotopes may have different electron affinity, and molecules such as N2 or O2, symmetric and nonpolar in their most common form, may have trace isotopologues that are slightly polar. The contribution of a molecular dipole μi on refractivity is quadratic, and varies as niμi2 (see equation (2)). Since isotopically asymmetric N2 and O2 have small abundance and will be only weakly polar, we will neglect isotopic effects on the optical properties of dry air.

3.5. Properties of Water Vapor

[31] The amount of water in moist air is expressed in this model as x, the molar fraction of water vapor in air. Since water is a major contributor to the refractivity, and is by far the largest contributor to the polar refractivity of air, it was particularly explored whether the isotopic composition of water could have any significant impact. Water is strongly polar, so it is the substance where an isotopic difference in the molecular dipole has the largest chances of being significant. In this study, water is split into an ensemble of several isotopologues. This includes the most common light water (1H216O), heavy-oxygen water (1H217O and 1H218O) and partially deuterated water (1H2H16O). The latter is optically the most different (see Table 2). It turns out however that although water is the substance where the isotopic optical differences are largest, among all substances considered here, they are still too small to be relevant given the small fractions of all but the common light water. Only the differences in mass are significant at the level of accuracy of this study, which means that all species, including water, can be safely considered as a pure substance whose molecular weight is the average by concentration of the different isotopologues [International Association for the Properties of Water and Steam (IAPWS), 2001].

Table 2. Isotopes of Water Considered in This Study, and Their Assumed Propertiesa
SubstanceMole FractionMassαμiχi

3.6. Linear Approximation

[32] In section 2, a theoretical model was prepared, including a full model, and its linear approximation in density. Within this linear approximation, the different contributions to the total refractivity by substance and process (electric polarizability, electric dipole, magnetic susceptibility) are separable. Although the aim of this study is to derive a more accurate expression, this linear approximation is useful to analyze the set of substances and physical processes that are relevant for the final model. The separable contributions in the linear approximation are shown in Table 3. The expression of the refractivity proposed in this work is not exactly linear. However, the nonlinearity does not change substantially the size of the contributions toward the total refractivity.

Table 3. Dry Air Linear Contributions to Refractivitya
SubstanceElectric NonpolarElectric PolarMagnetic
  • a

    Conditions representative of the Earth's surface are chosen: temperature of 15°C, pressure of 1013.25 hPa. The contributions are split in electric (related to εr) and magnetic (related to μr). Electric contributions are split into nonpolar, derived from the molecular polarizability, and polar, derived from the molecular electric dipole moments.


[33] It can be concluded from Table 3 that the contributions from dry air that are above the threshold of 0.01 N units at sea level are the neutral polarizabilities of N2, O2, Ar, and CO2, and the paramagnetic response of O2. As mentioned above, air contains these substances in different isotopologues. The presence of several isotopologues per substance has an impact in the density of air, which is above the threshold of accuracy for this study. However, the difference in the optical properties between isotopologues is not above the threshold.

[34] Table 4 shows the contributions to refractivity from water vapor that can be above the threshold of 0.01 N units at sea level conditions. Magnetic susceptibility is always negligible for water vapor. The amount of isotopologues is sufficient to lead to a significant fraction of refractivity produced by some form of heavy water, especially the heavy-oxygen water 1H218O. As for dry air, the actual composition has a nonnegligible impact in the value of the molecular weight of water. However, the optical properties of the isotopologues are very similar, and their difference can be ignored for the target accuracy of this study. All isotopologues of water are thus treated as optically identical to the most common 1H216O. The relevant EM parameters of water for this model are thus the molecular electric polarizability and the molecular electric dipole of 1H216O. Accurate values for both are provided by IAPWS [2001].

Table 4. Water Vapor Linear Contributions to Refractivitya
SubstanceElectric NonpolarElectric PolarMagnetic
  • a

    Conditions representative of the Earth's surface in warm and wet weather are chosen: temperature of 25°C, pressure of 1013.25 hPa. The contributions are split as in Table 3.


3.7. Relevant Parameters

[35] By selecting the parameters that, in the first approximation above, are related to contributions greater than 0.01 N units at typical surface conditions, the entire list of required input parameters is reduced to a few. All others can be safely discarded as irrelevant, except in case any of them contained a gross error in order of magnitude. This reduces the number of relevant substances to N2, O2, Ar, CO2, and H2O. Table 5 presents the reduced list of parameters pk, and their estimated uncertainties according to their respective sources.

Table 5. Relevant Parameters for a First Principles Model of the L Band Refractivity of Moist Aira
Fitting coefficients  a1a2a3a4
  • a

    The parameters have been selected from a broader model, as those whose value is important if the target accuracy is 0.01%. Values of 0 represent negligible dependences.

Total error  222.6820.0696701.6056385.886
Parameter pjValueError equation imagejequation imagejda1/dpjequation imagejda2/dpjequation imagejda3/dpjequation imagejda4/dpj
N2 dry air fraction0.780840.000010.00007000
O2 dry air fraction0.2094760.00001−0.00007000
Ar dry air fraction0.0093320.0000030000
CO2 dry air fraction0.0003850.0000050000
N2 polarizability1.73940.00010.0071000
O2 polarizability1.56960.00010.0009000
Ar polarizability1.64190.00010.0001000
CO2 polarizability2.91280.0010.00001000
O2 magnetic susceptibility3677.10.00.00600
H2O polarizability1.4940.001000.3130
H2O electric dipole1.854980.00005000.6370.637
Fitted values  0.0080.0060.80.64

4. The Proposed Expression

[36] In section 2.3, theoretical considerations led to the choice of a functional form for a practical expression of refractivity, using equations (11) and (12). The expression contains four parameters a1, a2, a3, and a4, ultimately related through a microphysical model to several molecular properties, which have to be determined by measurements.

[37] The determination of the values a1…4 is made in this study creating a wide sample of realistic atmospheric conditions, evaluating the refractivity with the microscopic physical model described by equations (1), (2), and (6), and fitting the parameters to optimally represent the evaluated refractivities.

[38] The sample is prepared with a number of vertical profiles of atmospheric properties. The profiles represent the Earth's atmosphere from the surface up to h = 32 km (pressure about 103 Pa), at 250 m intervals. Each profile follows the vertical temperature gradient dT/dh of a US Standard Atmosphere [NOAA et al., 1976], which in K/km is

equation image

Once the gradient is fixed, the entire temperature profile is determined by any fixed temperature point, for instance the surface temperature Ts. The sample includes a wide range of temperatures by choosing several profiles, whose surface temperature is Ts = −50°C, −40°C, …, +40°C. A range of moistures is also chosen, from perfectly dry to saturated in water vapor, at relative humidities 0%, 10%, …, 100%. To avoid unrealistic moisture contents in the stratosphere, the moist fraction is limited to never grow with altitude within a profile.

[39] The coefficients a1…4 are then fixed to optimally represent the sample. It was then verified that the parameters obtained are robust against modifications of the fitting sample, as long as the sample continues to be a wide representation of realistic conditions in the Earth's atmosphere.

[40] With the stated values of microscopic properties, and the sample mentioned, the best fit is

equation image

in units of N units/(kg m−3).

[41] The coefficients are determined assuming mean molecular weights according to the concentrations and weights in Tables 1 and 2. Therefore, the mean molecular weights of dry air and water vapor have been assumed to be md = 28.9655 and mw = 18.0153 (g/mol), respectively. It would be straightforward to reevaluate the fit if other average molecular weights were assumed in the relationship between mass and mole densities. The coefficients a1 and a2 would scale as md−1, whereas a3 and a4 would scale as mw−1.

[42] The fit reproduces the underlying microphysical model with maximum relative errors of 10−5 (fractional error in refractivity), and RMS relative error through the entire sample of 2 · 10−7. It is important to note that this is the accuracy of the fit representing the underlying model. The fitting accuracy should not be confused with the accuracy of the underlying model representing the actual behavior of moist air.

4.1. Estimated Accuracy of the Proposed Expression

[43] The physical model that has been constructed in the previous sections contains a number of input parameters, related to microscopic molecular properties. In Table 5, a selection of these model parameters was made, restricting the list to those that could affect the refractivity above certain threshold. We will call this list {p}. The physical model was then reduced to an expression containing 4 coefficients, which were called {a}. A more detailed error analysis is made here with the selected list of physical parameters, and their impact on the four parameters of the proposed expression.

[44] The analysis is made finding the derivatives dai/dpj, of the fitting coefficients with respect to variations of the physical parameters. These derivatives can translate the input errors image into the space of the coefficients {a}. This is also shown in Table 5. The derivatives with respect to the fractions are small, and have a negligible impact after considering the errors. As can be expected, the parameters that most limit the final accuracy are the polarizabilities of N2 and O2 for dry air, and the polarizability and dipole moment of water. The original measurements quote very good formal accuracies, and these translate to a formal error of 4 · 10−5 for the final expression. However, as quoted by those same sources, most of the measurements may contain systematic errors, not represented by the formal error. Following the same sources, a safer value of the relative error, allowing for systematic errors, is 1 · 10−4, which is adopted here.

[45] It is interesting to note that the dependence of the fit with respect to the value of the concentration of CO2 is very small. Indeed, the polarizability of a molecule of CO2 is substantially larger than that of a molecule of N2 or O2. Therefore, the polarizability per unit mol or per unit pressure changes with the concentration of CO2. However, the polarizability per unit weight is very similar, so a density-based expression changes negligibly. On the other hand, a varying value of CO2 does have an impact in the equation of state, and therefore the density of a parcel of air at a given pressure and temperature.

5. Review of Expressions in the Literature

[46] Refractivity expressions for the atmosphere in common use [Smith and Weintraub, 1953; Thayer, 1974; Bevis et al., 1994; Foelsche, 1999; Rüeger, 2002] are parameterized functions of pressure and temperature. Most include expressions of the form

equation image

where Pd and Pw are the partial pressures of dry air and water vapor, and T the absolute temperature. Thayer [1974], however, introduces compressibility factors Zd and Zw for dry air and water vapor, respectively.

equation image

[47] In all cases, the expressions suggest that the refractivity is assumed to be proportional to the densities of dry air and water vapor. For an ideal gas, which is assumed in most of the references cited, density is proportional to P/T. For a nonideal gas, density is proportional to P/TZ−1, which is the functional form chosen by Thayer [1974]. The parameters k1, k2 and k3 are assumed to be constants related to the properties of air and water. Their values are found by either fit to experimental values of refractivity or theoretical considerations related to other measurements.

[48] Although all these expressions correctly estimate the refractivity of air to a reasonable degree of accuracy (about 0.1%), none of them has the proper functional dependence if a slightly better accuracy is required. In general, there are two issues: first, dependence of the refractivity is assumed to be linear in density, ignoring nonlinearity, and second, nonideal behavior of air, which affects the relationship between pressure, temperature, and density, is either neglected or represented with insufficient accuracy.

[49] In order to compare against our proposed expression, we briefly review other expressions available in existing literature. Pressures are in hPa and temperatures T in K. Celsius temperature is expressed as t.

5.1. Smith and Weintraub [1953]


equation image

where SW53 denotes Smith and Weintraub [1953]. As Healy [2009] points out, Smith and Weintraub [1953] provide only a very approximate estimate of the CO2 contribution to refractivity, and approximate 0°C as 273 K. The CO2 content should besides be updated. Both introduce a small error, which is not negligible for the margin of tolerance that is targeted in this study. Smith and Weintraub [1953] find the difference between the molar polarizabilities of water and dry air negligible for atmospheric applications, and thus set k1 = k2.

5.2. Thayer [1974]


equation image

Thayer [1974] (TH74) chooses the compressibility factors Zd and Zw from Owens [1967],

equation image
equation image

These compressibility factors were intended by Owens [1967] to represent the behaviors of pure dry air (moistless) and pure water vapor (airless) and are compatible with the expression from Picard et al. [2008] that we have chosen for this study. However, Thayer [1974] assumes that these compressibility factors are applicable individually to the dry air and water vapor fractions that constitute the moist air mixture. In fact, the compressibility factor is a global property of a mixture, rather than of each individual substance, and is related to the interactions between molecules [e.g., Pathria, 1972]. In atmospheric air, where water vapor is always a small fraction, a molecule of dry air is mostly surrounded by other molecules of dry air. Zd, which had been measured with dry air, is still reasonably applicable to represent the behavior of the dry fraction of moist air. However, the Zw expression by Owens [1967] had been obtained with pure water vapor, with only water-water interactions. A molecule of water vapor in moist air is mostly surrounded by molecules of dry air (water-air interactions are dominant). Therefore the use of Zw for water vapor in moist air is not necessarily an accurate choice.

[52] Considering the functional form of the expression, and ignoring for now the values of the constants k1, k2, k3, the dry part of the expression of Thayer [1974] is reasonably well chosen, whereas the functional form of the moist terms are not necessarily better than an ideal-gas form such as that of Smith and Weintraub [1953].

5.3. Bevis et al. [1994]


equation image

where BE94 denotes Bevis et al. [1994]. This expression follows the functional form of Smith and Weintraub [1953] but allows one more degree of freedom to represent the different polarizabilities of air and water vapor, thus separating the expression in three terms. The choice of the function neglects nonideal-gas behavior. As noted by Healy [2009], approximates 0°C as 273K, as in work by Smith and Weintraub [1953], and neglect the impact of CO2.

5.4. Foelsche [1999]

[54] Although Foelsche [1999] (FO99) has not been in common use, presents a first-principles derivation of the refractivity of the air, which is interesting to compare with the present study. That derivation yields

equation image

Nonlinear dependencies on density and pressure, including dielectric enhancement and compressibility were considered, but only the linear contributions in pressure were retained. Therefore, the final expression is representative of the low-density limit. In that limit, the refractivity is slightly higher than in this model. Reviewing the input data of molecular polarizability, we find that this is mostly due to the assumed polarizability of O2, which derived ultimately from Newell and Baird [1965], and is higher than the value in this study. Newell and Baird [1965] had measured the EM properties of several gases at a frequency of 47.7 GHz. For O2, this is too close to a band of resonances, and is therefore not representative of L band frequencies. The MPM-93 model indicates that a measure of the polarizability of O2 at 47.7 GHz will be larger than in the static regime necessary for the L band. The excess suggested by MPM-93 is compatible with the difference in O2 polarizability between Newell and Baird [1965] and the values chosen for this study, from May et al. [2008], which derive from measurements obtained at a lower frequency (8 GHz).

5.5. Rüeger [2002]

[55] Rüeger [2002] (RU02) reviews several expressions, notably those of Smith and Weintraub [1953], Thayer [1974] and Bevis et al. [1994] and finds that in all the expressions some inappropriate approximation had been made. The compressibility factor and the CO2 composition of dry air are mentioned among the contributions that are nonnegligible. However, Rüeger [2002] proposes a new expression whose functional dependence is chosen as P/T, both for dry air and water vapor, which would be appropriate for ideal gases, but that cannot capture all the thermodynamic behavior of a nonideal gas, recognized as relevant. The coefficients presented are the result of a fit to experimental values,

equation image

The fit allows some implicit inclusion of the compressibility factor. However, this is constrained by the functional dependence chosen for the refractivity, which can only partially represent the compressibility. Even if we assumed that the input data were perfect, the expression would be bound to be accurate only at conditions in the vicinity of the measurements. In GPSRO, where expressions have to be used over a very wide range of pressures, temperatures, and moisture contents, the choice of this functional form could be inaccurate far from the fitting region.

6. Comparison of the Different Expressions

[56] A direct comparison of the different expressions is not trivial. All expressions in section 5 use the concept of partial pressure, to indicate the amounts of dry air and water vapor present in the moist air mixture. However, unlike the total pressure of the mixture, the partial pressures of the fractions are not directly measurable quantities, but theoretical constructs, well defined only in the ideal-gas case. In a nonideal-gas mixture, partial pressures could be defined as an extension of the ideal-gas concept. There is not, however, a uniquely defined way to perform this extension, which could be based on different measurable quantities, for instance, molar fractions, chemical potentials, fugacities, or others. The studies mentioned in section 5 either ignore nonideal-gas behavior or fail to specify any particular extension of the concept of partial pressure.

[57] Among other issues, partial pressures in a nonideal mixture may not be additive. The partial densities, instead, are still additive even if strong nonideal behavior is present. Therefore, in this work, the amount of the different fractions is expressed in the form of partial densities.

[58] Since an unambiguous meaning of partial pressure was not specified, the expressions in section 5 are only well defined in either the ideal-gas limit (low density) or in a nonmixed state, where partial pressure equals total pressure.

[59] In order to illustrate this inaccuracy in the definition, we choose two possible technical definitions of the partial pressures of dry air and water vapor for a given sample of moist air. Given a partial density of water q = ρw/ρ, which is a precisely defined quantity, or equivalently a water mole fraction xw, several choices are possible, among them the following two.

[60] 1. In choice a, the partial pressures are proportional to the respective mole fractions of dry air and water vapor, for a total pressure P. The partial pressures are Pda = P · (1 − xw) for dry air and Pwa = P · xw for water vapor.

[61] 2. In choice b, find a state of pure dry air whose density is ρ · (1 − q), at the same temperature (the initial sample, dehydrated). It will have a pressure Pdb. We define the partial pressure of water vapor as Pwb = PPdb.

[62] There is no reason to choose one over the other, and other definitions are possible. Any practical use of these expressions will somehow make an implicit choice of a definition for the partial pressures, given other quantities. The relevant issue is that these are all identical under ideal-gas conditions, but not for a real gas, leaving the concept of partial pressure ambiguous. By construction, all definitions fulfill P = Pda + Pwa = Pdb + Pwb. In Figure 2, the difference between both definitions is shown for realistic conditions of atmospheric pressure and temperature. This indicates that all the expressions in section 5 are not only subject to inaccuracies in the parameters ki, but they also carry an intrinsic uncertainty, linked to the inaccurate definition of their respective arguments (i.e., the partial pressures). This uncertainty is on the order of 0.3% in the moist refractivity, when applied to conditions in the low troposphere. This also leads to the inaccurate definition of the associated dry pressure of a mixture, and the associated dry refractivity. This structural inaccuracy in the definition of the Pd and Pw arguments is not only an issue when an expression is used, but potentially also when the parameters of an expression are being determined from experimental data.

Figure 2.

Relative difference between two possible interpretations of the concept of partial pressure of water vapor, as specified in section 6.2, for given states of pressure, temperature, and moisture content. To show realistic sizes under atmospheric circumstances, the conditions of pressure and temperature are those from US Standard Atmosphere profiles of surface temperatures TS = 20°C (solid line), TS = 30°C (dotted line), and TS = 40°C (dashed line). The water vapor content is that at saturation.

6.1. Dry Air

[63] In the absence of water vapor, it is possible to make a realistic direct comparison of the refractivity across the different expressions. Air in the stratosphere, and in the troposphere in desert or polar regions, has very small amounts of moisture. The comparison of the dry terms of the different expressions is meaningful under these conditions. This is shown in Figure 3. Figure 3 shows the quantity NT/P for dry air along several isotherms, for conditions relevant to the Earth's troposphere and stratosphere. This quantity is constant according to models SW53, BE94, FO99 and RU02. A small, but nonnegligible dependence appears with respect to pressure and temperature in TH74 and in the proposed model, equations (12) and (11), with coefficients (15). This is due to the introduction of compressibility factors in TH74, and additionally to other nonlinear dependencies in the proposed model. The nonlinear behavior, including the compressibility, is comparable to the difference between the models considered. It is also clear that the quantity NT/P is only approximately constant (note that the y axis of Figure 3 is highly magnified).

Figure 3.

Comparison of the dry air refractivity among the different models mentioned, including this work. Shown is NT/P of dry air, as a function of pressure, along several isotherms: −30°C, 0°C and 30°C. For reference, in the Earth's atmosphere: log10P ≃ 5 at MSL, log10P ≃ 4 at an altitude on the order of 15 km, and log10P ≃ 3 at an altitude on the order of 30 km.

[64] In order to illustrate the impact under typical atmospheric circumstances, the same comparison is made as a function of altitude for several atmospheric profiles. A crude but representative vertical profile for Earth's atmosphere can be given by the US Standard Atmosphere. Several representative profiles can be obtained by fixing the vertical temperature gradient and selecting several surface temperatures. The comparisons are shown in Figure 4. Figure 4 shows a change in pattern around 11 km, the location of the tropopause according to the chosen model. Below that level the temperature reduces with altitude, with a gradient of about −6.5 K/km. Above that level and for most of the rest of the diagram, the atmosphere has a nearly constant temperature.

Figure 4.

Comparison of the dry air refractivity among the different models mentioned, including this work, when applied to realistic profiles of atmospheric temperature. Shown is NT/P of dry air, as a function of altitude, along profiles that follow the US Standard Atmosphere vertical temperature gradient. Shown are two profiles of surface temperatures −30°C and 30°C. Models SW53, BE94, FO99, and RU02 (several long-dashed and dash-dotted types) have constant NT/P. Both TH74 (short-dashed lines) and this model (solid lines) show variable NT/P, which is larger at lower altitude. For both models, the colder profile shows a larger NT/P.

[65] The differences between expressions that are shown in Figures 3 and 4 are small, but it is known (see section 1) that details of this size are nonnegligible for NWP applications if GPSRO is to be considered as an unbiased data source.

6.2. Moist Air

[66] In contrast to dry air, water vapor is not found nearly pure in the atmosphere, but always as a small fraction in moist air. A realistic comparison of the behavior of the moist terms in the different expressions cannot be easily disentangled from the behavior of the dry terms, although the classical three-term expressions (17) suggest that they are entirely independent.

[67] We therefore compare the increment in refractivity as a parcel of air is moistened at constant P, T from a dry state, across the different expressions. The dry air is partially displaced by the more refractive water vapor molecules, resulting in an increase of the refractivity. In Figure 5, the increment in refractivity is shown for the 30°C isotherm, at different pressures. The increment is normalized by the partial density of water in the moist state. Since the available expressions do not specify an interpretation of the partial pressures, they are not unique. The two possible interpretations mentioned above are shown. For tropospheric pressures, where moisture is relevant, the difference between the two interpretations of the same expressions is comparable to the difference between expressions, underscoring the importance of the structural uncertainty in their definition.

Figure 5.

Increment in refractivity with respect to dry air, as it is moistened at constant pressure, over the isotherm T = 30°C, for the different expressions considered. The increment is normalized to the water vapor density, and illustrates the moist terms in the different expressions. For each expression in section 5, two curves are plotted, representing two possible interpretations of the partial pressure, as mentioned in the main text.

[68] The spread between the different definitions for a given expression is partially due to the particularly strong nonideal effects of moisture, which are larger than for pure dry air, and that affect the assumed molar amounts of air and water. Not only does water vapor behave as a nonideal fraction in air, but it also enhances the nonideal behavior in the dry fraction that is interacting with it. Under ideal-gas conditions, when adding water at constant P and T to dry air there would be a one-to-one substitution of dry air molecules by water vapor molecules. This is maintained in the choice a above. But the density of dry air in a moist state at a given pressure is larger than that given by an ideal-gas approximation. Therefore the refractivity does not increase during moistening only because the new water molecules are more refractive than the dry molecules they displace. In a real gas, introducing cohesive molecules such as water vapor at constant pressure causes a substitution of less than one-to-one. Some of the dry molecules are not displaced and still contribute to the total refractivity in the moistened state. This is represented in choice b.

6.3. The Robustness of a Three-Term Pressure Formula

[69] Regardless of the accuracy of the microscopic parameters chosen in this study, it is also possible to analyze the impact of choosing another expression as a fitting ansatz. As an illustration, we can perform the fitting of the same microscopic model, but against a three-term pressure expression, as (17). As mentioned, the values of the partial pressures, given a mixture of moist air, are not unambiguously defined. We can choose either the interpretation of choice a or choice b (see above) of the meaning of partial pressures of dry air and water vapor, in each case obtaining different fits. In the first, we would get, in units of hPa and K,

equation image

whereas with the second choice we would get

equation image

Besides having different results depending on the interpretation of the meaning of partial pressure, the residuals of the best fits are of 0.1% rms, and with maximums of 0.2%, within realistic atmospheric conditions. This is 2 orders of magnitude larger than the residual that can be obtained over the same data with the functional form proposed in this study, and of a size recognized to be nonnegligible for NWP purposes [Aparicio et al., 2009; Healy, 2009; Cucurull, 2010]. Moreover, the values of the best fit coefficients are not robust against the choice of the fitting sample. Most notably, a sample containing only the low troposphere has a larger value of k1. For instance, if only the subsample of pressures greater than 500 hPa is used, we obtain k1 = 77.637, and if greater than 800 hPa, k1 = 77.651.

7. Conclusion

[70] The atmospheric refractivity at L band radio frequencies was reviewed. A first principles microphysical model was developed, that links L band refractivity and accurate measurements performed below 10 GHz with each of the gases in the atmosphere. The impact of each parameter was quantified. The accuracy of these measurements translated to an approximate relative accuracy on the order of 0.01% for the microphysical model, thus fulfilling the objective. A practical expression was fitted to this model, namely

equation image


equation image

with ρd and ρw the densities of dry air and water vapor (kg/m3), and τ = 273.15/T − 1, with T in K. The coefficients were found to vary negligibly with the CO2 content of the air, as the polarizability per unit mass of CO2 is very close to that of N2 and O2. The expression was found to be well behaved, robust against the selection of the fitting sample, and represented the behavior of the underlying microphysical model with an accuracy on the order of 0.001% over a wide sample of realistic atmospheric conditions, sufficient enough to not degrade the model's estimated accuracy of 0.01%.

[71] This expression was compared to other available expressions in the literature. Due to limitations in their respective functional forms, they were found inadequate to deliver an accuracy better than 0.1% rms, and with 0.2% maximum errors, within the context of an application targeting the entire range of pressure, temperature and moisture found in Earth's tropospheric and stratospheric conditions, regardless of the exact value of the parameters. This is in particular the case with applications in GPS radio occultations. A modification of the functional form, and not just the values of the coefficients, is clearly necessary to reach a superior level of accuracy. The proposed expression is expected to be relevant for applications where absolute accuracy of the refractivity is particularly important. The range of applicability is for all frequencies below 10 GHz.


[72] We wish to thank Michael Moldover (NIST) for providing very useful comments concerning the measurement of the dielectric properties of atmospheric gases. We also thank Sean Healy (ECMWF) and Lidia Cucurull (NOAA) for sharing their experience testing the adequacy of different expressions of refractivity for NWP purposes and Louis Garand for useful comments on this manuscript.