Extinction, refraction, and delay in the atmosphere

Authors


Abstract

[1] This paper establishes that absolute optical air mass and hydrostatic atmospheric delay of electromagnetic waves are proportional magnitudes, and, consequently, their respective obliquity ratios are identical dimensionless quantities. This means that a potential source for developing new models for relative optical air mass can be found in the formulae for the atmospheric delay in electromagnetic signals (and vice versa). In this respect, for estimating relative optical air mass, we demonstrate that Herring's (1992) family of mapping functions for modeling atmospheric delay is more accurate than functional forms devised expressly for the purpose, such as that of Kasten (1965).

1. Introduction

[2] More than two centuries ago Laplace [1805, p. 284] established that for an atmosphere with spherical symmetry whose density decreases exponentially with height, the logarithm for the intensity of incoming light from any heavenly body is proportional to its refraction divided by the cosine of its apparent elevation angle. According to Young [2006], this implies that

equation image

where r is the refraction of the light incident at an apparent zenith distance za and m is the air mass (relative optical air mass or obliquity ratio of the absolute optical air mass).

[3] The fact that relation (1) continues to hold true, at least in approximate terms, with atmospheric models more verisimilar than the exponential one [Young, 2006] has meant that recognized tables and formulae for estimating either of the magnitudes, whether atmospheric refraction or air mass, have been advantageously applied to the development of models for calculating the other magnitude [e.g., Forbes, 1842; Kristensen, 1998].

[4] On the other hand, in the field of radar tracking of objects in space from the Earth's surface, Rowlandson and Moldt [1969] showed that, assuming an atmosphere of spherical symmetry whose refractivity decays exponentially with height,

equation image

where r′ is the refraction of radio waves arriving at the tracking station with an apparent elevation angle ɛa and a delay D. As was the case by virtue of relation (1), relation (2) has likewise prompted that formulae designed to estimate either of the variables in question, whether atmospheric refraction or atmospheric delay, have been successfully applied to the development of models for calculating the other variable [e.g., Marini, 1972; Yan, 1996].

[5] Given that both light and radio signals are electromagnetic waves and that the elevation angle and zenith distance are complementary angles, it is remarkable that it has not been previously observed that for the same wavelength Dm or its equivalent

equation image

where M is the absolute optical air mass. The closest statement has been given by Young [2006, p. 111], who notes that “the tropospheric corrections required in GPS calculations are more closely allied to air mass than to refraction.”

[6] This paper establishes that absolute optical air mass and hydrostatic atmospheric delay are proportional magnitudes, and, consequently, their respective obliquity ratios are identical dimensionless quantities. From this it can be demonstrated that the known formulae for calculating the atmospheric delay represent a good basis for the purpose of developing new models for the estimation of absolute optical air mass (and vice versa). To this end, we have found that when estimating air mass, the family of mapping functions for modeling atmospheric delay presented by Herring [1992] surpasses, among others, the model proposed expressly for the purpose by Kasten [1965]. Finally, the link established between absolute optical air mass and signal delay has allowed us to express the relation between this latter variable and its own atmospheric attenuation as a simple mathematical relation.

2. Relation Between Optical Air Mass and Atmospheric Delay

[7] According to Bemporad [1904, p. 5], absolute optical air mass is defined as

equation image

where S represents the trajectory followed by monochromatic stellar light, s denotes the arc length from the initial point of S, and ρ(s) is the air density along S. On the other hand, all electromagnetic waves arriving at the Earth from any particular source undergo a delay in reaching the surface with respect to the time that they would take in the absence of an atmosphere. This time delay is termed atmospheric delay and can be expressed in terms of the distance covered by light in empty space, thus,

equation image

where n(s) is the refractive index of the wave, LS is the trajectory length, and LE is the Euclidean distance between the source and the receptor [Dodson, 1986].

[8] If we take into consideration that the so-called geometric delay, δ = LSLE, is a small fraction of the total atmospheric delay D, even for zenith distances near the horizon [Freeman, 1962; Yan and Ping, 1995], we obtain the well-known relationship

equation image

If, as in work by Kasten [1965], Garfinkel [1967], and Foelsche and Kirchengast [2001], it is assumed that for dry air

equation image

where the specific refractivity of the air k is considered a constant, the result is

equation image

which proves relation (3).

[9] Moreover, if the obliquity ratio of the atmospheric delay is defined as

equation image

where ɛ is the elevation angle and Dz = D(90°) = kequation imageρ(sz)dsz is the atmospheric delay of the wave following the straight trajectory Sz from the zenith, and the obliquity ratio of absolute optical air mass is, in its turn,

equation image

with Mz = M(90°) = equation imageρ(sz)dsz, by virtue of equation (8) we obtain

equation image

That is to say, the obliquity ratio of the hydrostatic atmospheric delay and the air mass are two dimensionless quantities, which are practically identical, as required.

[10] This result is highly significant because it lays the foundations to using any family of mapping functions for atmospheric delay, fd(ɛ), as a functional form for approximating the air mass, fm(ɛ), and vice versa. Put in other terms, if we start from the hypothesis that fd(ɛ) ≈ d(ɛ) and fm(ɛ) ≈ m(ɛ), then it is also true that fd(ɛ) ≈ m(ɛ) and fm(ɛ) ≈ d(ɛ).

[11] It should be noted that the wavelengths employed in satellite laser ranging (SLR) are in the ultraviolet (355 and 423 nm), visible (532 and 694.3 nm), and infrared (847 and 1064 nm) ranges of the electromagnetic spectrum [Mendes and Pavlis, 2004]. At these wavelengths the air behaves as a dispersive medium, for which reason the delay of SLR signals are worked out from the group refractive index instead of from the phase refractive index [Sinclair, 1982]. Therefore, in the case of SLR signals, the constant k in equation (8) refers to the group-specific refractivity.

3. Functional Forms to be Compared

[12] In order to put these findings to the test, four commonly used families of approximation functions dependent on three parameters were selected, namely, the formulations proposed by Kasten [1965], Marini [1972], Herring [1992], and Gueymard [1993]. It should be noted that while the first and fourth of these were developed specifically to estimate air mass, the second and third were designed to estimate electromagnetic signal delay through the atmosphere. It should also be made clear that in sections 3.13.4, the vector a symbolizes the set of parameters ai (i = 1, 2, 3).

3.1. Kasten's Formulation

[13] This functional form was originally proposed by Kasten [1965] and was later recalibrated by Kasten and Young [1989]. Its mathematical expression is

equation image

[14] As can be inferred from work by Gueymard [2003], the formulae by Kasten [1965] and Kasten and Young [1989] are those most frequently incorporated into broadband solar radiation models; the most recent is also the equation preferred within the field of meteorology for estimating air mass [Kasten, 1993]. Finally, it is worth noting that Kasten [1965] also provided an approximation formula for relative optical mass for atmospheric water vapor on the basis of the same functional form.

3.2. Marini's Formulation

[15] Although Ifadis [1986, 2000] has been the first, and to date the only, researcher to adopt this formulation, the functional form which concerns us here actually corresponds to the triparametric version of the truncated continued fraction used by Marini [1972]. Its mathematical expression is

equation image

In his earlier work, Ifadis [1986, chapters 6 and 8] proposed mapping functions not only for hydrostatic but for nonhydrostatic (wet) atmospheric delay using this formulation.

3.3. Herring's Formulation

[16] Introduced as a variation of Marini's [1972] fraction mentioned in section 3.2, this triparametric family by Herring [1992] is the formulation chosen for most of the hydrostatic and wet mapping functions proposed afterward [e.g., Niell, 1996, 2000; Mendes et al., 2002; Boehm and Schuh, 2004; Boehm et al., 2006]. The mathematical expression is

equation image

Note that regardless of the value of a = (a1, a2, a3), after the modification introduced by Herring, it is always true that f(90°, a) = 1 = d(90°) = m(90°).

3.4. Gueymard's Formulation

[17] Gueymard [1993] based this triparametric family on Kasten's [1965] formulation with the aim of ensuring that f(90°, a) = 1. The mathematical expression is

equation image

where the elevation angle ɛ must be introduced in degrees. Gueymard [1993] also proposed approximation formulae for relative optical mass of both atmospheric water vapor and atmospheric ozone using his functional form.

4. Tabular Reference Values

[18] In order to check the adequacy of the given functional forms, we have taken two reference tables. The first one is the air mass table by Kasten and Young [1989], and the second one is the mapping table of hydrostatic delay by Chao [1972].

4.1. Kasten-Young Table

[19] Kasten and Young [1989, Table 2] present values for the obliquity ratio of optical air mass as a function of the apparent elevation angle of the incident radiation. It includes 336 entries with step widths of 0.1° between 0° and 20°, 0.2° between 20° and 30°, 0.5° between 30° and 55°, and 1° between 55° and 90°. On the basis of the air density vertical profile of the standard atmosphere ISO 2533 and a light refractive index at sea level of 1.000276, Kasten and Young [1989, Table 2] provided the numerical basis for the air mass approximation formula by Kasten and Young [1989].

4.2. Chao's Hydrostatic Table

[20] As with previous mapping tables for atmospheric delay by Chao [1971], his most recent version was presented as an internal document of the Jet Propulsion Laboratory [Chao, 1972] and did not become widely known by the scientific community until the work by Estefan and Sovers [1994, appendix] was published more than two decades later. The dry (hydrostatic) and wet tables by Chao [1972] provide values for the obliquity ratio for the atmospheric delay of radio signals as a function of the true elevation angle of the source. They offer step widths of 0.1° between 0° and 10° and 0.5° between 10° and 90°, such that each table contains 261 entries. Both tables were obtained from the atmospheric refractivity profiles inferred from data acquired during 1967 and 1968 by radiosonde balloons released from various globally distributed weather stations. Compared to previous versions, the essential improvement of these updated tables by Chao [1972] consisted in taking into account the small effect of signal path bending [Estefan and Sovers, 1994]. After more than 20 years of being used by NASA for determining orbits of spatial flights, the tables by Chao [1972] were superseded by a new generation of mapping functions [e.g., Davis et al., 1985; Ifadis, 1986; Herring, 1992; Niell, 1996], all of them featuring sensitivity to temporal variations of local atmospheric conditions. Nevertheless, the hydrostatic table by Chao [1972] is perfectly suited to the objective of this study and so is used here.

5. Degree of Approximation

[21] In order to compare the degree of approximation to the air mass given by the four chosen formulations, we adjusted the value of their respective parameters by the same procedure followed by Kasten [1965] and Kasten and Young [1989]. Namely, given a set of N values of air mass and the corresponding values of the elevation angle of the incident radiation, mj = mj) (j = 1, 2, …, N) and given the functional form f(ɛ, a), the procedure consists in determining the value of the components of a that minimize the relative root-mean-square error:

equation image

where ej(a) = [mjfj, a)]/mj.

[22] In contrast to the method described above, the adjustment of parameters from a given family of mapping functions is not usually carried out in terms of relative error but rather as absolute error [Niell, 2000]. That is, given N values of the obliquity ratio for the corresponding values of elevation angle of the signal source, dj = dj) (j = 1, 2, …, N) and given the zenith delay of the signal Dz and the functional form f(ɛ, a), the procedure consists in determining the value of the components of a which minimize

equation image

where Ej (a) = Dz [djfj, a)].

[23] As mentioned in section 4.2, the hydrostatic table by Chao [1972] provides the obliquity ratio of atmospheric delay for true elevation angles ranging between 0° and 90°. However, as the range of application of most modern mapping functions falls between 3° and 90° [e.g., Herring, 1992; Niell, 1996, 2000; Mendes et al., 2002; Boehm and Schuh, 2004; Boehm et al., 2006], any values in the table which fell outside this interval were discarded before adjustment of the parameters was applied. In addition, following certain observations by Estefan and Sovers [1994], some anomalous entries in Chao's [1972] hydrostatic table were detected and dismissed, namely, the values corresponding to true elevation angles 5.1°, 9.1°, 9.2°, 9.3°, and 11.0°. Hence, after truncating and slightly filtering Chao's [1972] hydrostatic table, the adjustment of the function parameters was undertaken on 226 entries of the obliquity ratio of atmospheric delay. Finally, following the work by Niell [1996], we adopted 2.3 m as the nominal value for zenith hydrostatic delay.

[24] It should be noted that so far ɛ symbolized indistinctly both the apparent and the true elevation angle of the signal source. Henceforth, when necessary, the apparent elevation angle will be represented by ɛa (as in section 1) and the true elevation angle will be represented by ɛt.

6. Numeric Results

[25] Table 1 displays, for each of the four families of approximation functions considered, the essential features of the adjustment performed on the values of air mass from the table by Kasten and Young [1989]. While the third column contains the optimal vectors ao that minimize the root-mean-square relative error rmse(a), the corresponding values of that error, rmsem = rmse(ao), are recorded in the second column. Figure 1 reflects, as a function of the apparent elevation angle, the relative error of each formulation once the respective optimal vectors have been adopted, namely ea, ao). As can be observed in the plots, the maximum relative deviation is always located at the origin of the definition range, i.e., ɛa = 0°. The numeric values of the corresponding relative error, e′ = e(0°, ao), are presented in the fourth column of Table 1. Taken together, Figure 1 and Table 1 allow us to infer that the triparametric formulations of Marini [1972] and Herring [1992], originally proposed as families of mapping functions for atmospheric delay, can also provide good approximations of air mass. In this respect, the functional form by Herring deserves special mention as its results clearly surpass those of the formulations designed for the specific purpose of air mass approximation by Kasten [1965] and Gueymard [1993].

Figure 1.

Relative error of the fits for the three-parameter forms to data of the air mass table by Kasten and Young [1989]. Plotting, as a function of apparent elevation angle ɛa, of the relative error of fit ea, ao) provided when the optimal vector ao is adopted for each formulation. Note that we are just representing the part of the interval of definition where the error becomes more oscillating and, therefore, where deviation reaches its highest values.

Table 1. Results of the Fits for the Three-Parameter Formulations to Data of the Air Mass Table by Kasten and Young [1989]a
Formulationrmsem (%)aoe′ (%)
  • a

    Here rmsem = rmse(ao) is the minimum value of root-mean-square relative error provided by the optimal vector ao and e′ = e(0°, ao) is the relative error at the starting point of the application range of the formula, where the relative deviation is maximal. The numerical values of the components of ao for the formulations by Kasten [1965] and Gueymard [1993] are expressed in degrees.

Kasten [1965]0.0675.05721 · 10−10.432
6.07995 
1.63644 
Marini [1972]0.0931.03577 · 10−3−0.316
3.26178 · 10−3 
8.24226 · 10−2 
Herring [1992]0.0271.06607 · 10−3−0.169
3.69171 · 10−3 
9.08646 · 10−2 
Gueymard [1993]0.0853.08363 · 10−30.512
5.36281 
1.40096 

[26] Regarding the obliquity ratio of the hydrostatic atmospheric delay, the reference values for which were obtained from the hydrostatic table by Chao [1972], the third column in Table 2 records the optimal vector ao, minimizing the root-mean-square error RMSE(a) for each of the families of approximation functions that were compared. The numeric values of that error, RMSEm = RMSE(ao), are included in the second column of Table 2. Figure 2 shows, as a function of the true elevation angle, the error of each formulation once the respective optimal vectors, namely Et, ao), have been adopted. As can be observed in the plots, the maximal deviation is always located at the origin of the definition range, namely ɛt = 3°. The numeric values corresponding to this error, E′ = E(3°, ao), are presented in the fourth column of Table 2. Figure 2 and Table 2 illustrate that the triparametric formulations by Kasten [1965] and Gueymard [1993], originally proposed as approximation forms for air mass, also provide acceptable results for hydrostatic atmospheric delay.

Figure 2.

Error of the fits for the three-parameter forms to data of Chao's [1972] dry mapping table. Plotting, as a function of true elevation angle ɛt, of the error of fit Et, ao) provided when the optimal vector ao and a nominal zenith delay of 2.3 m are adopted for each formulation. Note that we are just representing the part of the interval of definition where the error becomes more oscillating and, therefore, where deviation reaches its highest values.

Table 2. Results of the Fits for the Three-Parameter Formulations to Data of Chao's [1972] Dry Mapping Tablea
FormulationRMSEm (mm)aoE′ (mm)
  • a

    RMSEm = RMSE(ao) is the minimum value of root-mean-square error provided by the optimal vector ao, and E′ = E(3°, ao) is the error at the starting point of the application range of the formula, where the deviation is maximal. The errors are obtained with respect to a nominal value for zenithal delay of 2.3 m, and the numerical values of the components of ao for the formulations by Kasten [1965] and Gueymard [1993] are expressed in degrees.

Kasten [1965]1.142.99515 · 10−13.09
4.82021 
1.41961 
Marini [1972]3.671.20484 · 10−3−7.18
1.95349 · 10−3 
3.51647 · 10−2 
Herring [1992]0.271.26018 · 10−3−1.27
2.97396 · 10−3 
6.52916 · 10−2 
Gueymard [1993]1.341.67740 · 10−16.51
3.72468 
1.15453 

[27] Note that the adjustment for air mass approximation was carried out in terms of relative error and apparent elevation angle (starting at 0°), whereas that for hydrostatic atmospheric delay (including geometric bending) was applied in terms of absolute error and true elevation angle (starting at 3°). However, such dissimilarities that can be considered circumstantial do not prevent two important coincidences from standing out: first, the four formulations considered have shown themselves to be adequate for approximating the two magnitudes in question, and second, the numeric results obtained with the functional form by Herring [1992], have, in both cases, clearly been better than those of the other forms in the comparison. Remember that the formulations by Herring and Gueymard [1993] appeared at their time as slight modifications of the functional families by Marini [1972] and Kasten [1965], respectively, in order to insure the zenithal convergence to unity. Nevertheless, the collateral effects of this retouching turned out to be different: the functional form by Gueymard [1993] obtains zenithal convergence at the expense of a worse approximation for elevation angles near the horizon, whereas Herring's improvement near the zenith generalizes throughout the definition range.

[28] Herring's [1992] continuous truncated fraction combines two qualities which give it an advantage over other functional forms. First, the number of fitting parameters is discretionary, and second, for a given vertical profile of atmospheric density, the precision of the statistical adjustments is highly sensitive to the named number of parameters. In order to confirm that Herring's fraction has the latest property (the first is self-evident), we repeated the fitting procedure described in section 5 with the four-parameter version:

equation image

where a = (a1, a2, a3, a4). The results are shown in Table 3, where it is verified that the minimized value of the root-mean-square relative error and the maximum relative deviation are more than 10 times smaller than in the case of the three-parameter form by Herring (see Table 1). Interestingly, it is worth noting that a formula based on Herring's four-parameter family has already been used by Ifadis and Savvaidis [2001] as a mapping function in the context of satellite geodesy (having the true elevation angle as argument).

Table 3. Results of the Fits for the Four-Parameter Formulation by Herring [1992] to Data of the Air Mass Table by Kasten and Young [1989]a
Formulationrmsem (%)aoe′ (%)
  • a

    Here rmsem = rmse(ao) is the minimum value of root-mean-square relative error provided by the optimal vector ao and e′ = e(0°, ao) is the relative error at the starting point of the application range of the formula, where the relative deviation is maximal.

Herring [1992]0.00251.03774 · 10−30.0115
 2.16438 · 10−3 
 7.50967 · 10−3 
 1.36978 · 10−1 

[29] Creating approximation formulae for air mass (as well as mapping functions for atmospheric delay) can be done using either apparent angular distances (whether apparent elevation angle or apparent zenith distance) or true angular distances (whether true elevation angle or true zenith distance). In fact, choosing one option or the other as argument will depend on the context in which the formulae are to be applied, namely, the angular distance available in each case [Young, 1994]. Hence, whereas some approximation formulae for air mass are created as a function of true angular distances [e.g., Young and Irvine, 1967; Young, 1994], others (the majority) are defined as a function of apparent angular distances [e.g., Hardie, 1962; Rozenberg, 1963; Kasten, 1965; Kasten and Young, 1989; Gueymard, 1993, 1995; Kristensen, 1998]. Young [1974, Figure 12a] plotted the error introduced by using true zenith distance as the argument in place of apparent zenith distance for calculating air mass by means of Bemporad's [1904, Appendix 1] tables. This error increases the further one goes from the zenith (where it is zero), and it becomes greater than 10% near the horizon [Young, 1994]. Consequently, when using a specific approximation formula for air mass, attention must be paid to ensure that the appropriate argument is being used.

7. Discussion and Conclusions

[30] We have demonstrated that the hydrostatic atmospheric delay of an electromagnetic signal is (practically) proportional to the absolute optical air mass of its trajectory, with the constant of proportionality being the specific refractivity of the air for the signal frequency in question. From this, it can be deduced as a corollary that the obliquity ratios of hydrostatic atmospheric delay and absolute optical air mass are two (almost) identical dimensionless quantities.

[31] The demonstrations above entail that when space geodesists endeavor to find an approximation formula for the atmospheric delay to electromagnetic signals or atmosphere scientists attempt the same with respect to air mass, each of them is dealing with different (but related) aspects of physics. Nevertheless, following the main thesis of this work, both research communities essentially face the same mathematical problem, such that ideas and methods developed to solve it by one community can be applied to the other community. In this respect we concur with Young [2006] when he states that scientists working on tropospheric corrections for GPS ought to have a good understanding of the meaning of air mass; although, as we have seen in the exposition above, the reciprocal is equally true, namely, researchers looking into atmospheric extinction of light should be aware of advancements in the study of atmospheric delay of signals.

[32] In science, any theoretical unification, as modest as it may be, normally brings with it various practical implications. Given that it is impossible to predict what all of these might be, we will suggest only those which are the most evident and promising.

[33] In principle, any basic formulation of a mapping function for hydrostatic atmospheric delay can be used as a parametric approximation model for air mass (and vice versa). In this regard, we have verified, on one hand, that the formulations by Kasten [1965] and Gueymard [1993], produced for estimating air mass, are also adequate as mapping function families for hydrostatic atmospheric delay and, on the other hand, that the formulations by Marini [1972] and Herring [1992], originally proposed for estimating atmospheric delay of radio signals, are likewise suitable as functional forms for approximation of air mass. Moreover, the formulation by Herring proved to be clearly better than all other parametric models compared, either for atmospheric delay or air mass.

[34] Young [1994] attempted to obtain an approximation formula for air mass combining, among others, the requirements of converging to unity at the zenith and giving higher precision than the formula by Kasten and Young [1989]. For this purpose, Young [1994, equation (5)] developed a six-parameter function on the basis of the numeric table by Kasten and Young [1989]. Although Young [1994] adopted the true zenith distance zt as argument, the precision of his formula is directly comparable to the accuracy of the formulae obtained in this work. As a result of such comparison, it can be observed that when adopting Herring's [1992] fraction, four parameters are sufficient to obtain a maximum relative deviation smaller than the six-parameter formula by Young, namely, 0.0115% (for ɛa = 0°, see Table 3) against 0.03% (for zt ≈ 63° [Young, 1994]).

[35] Gueymard [1995] also pointed out the desirability of finding a functional form that would combine the above-mentioned requirements by Young [1994]. Hence, he introduced a new four-parameter family that enables one to obtain approximation formulae for relative optical masses of the atmospheric constituents which disperse or absorb solar radiation. The author himself calibrated the parameters in question in the cases of dry clean air (Rayleigh atmosphere), ozone, water vapor, nitrogen dioxide, aerosols, and other substances [Gueymard, 1995, 2003; Gueymard and Kambezidis, 2004]. As we were able to confirm, formulae based on Herring's [1992] continuous truncated fraction proved to be robust and accurate in approximating air mass (Rayleigh atmosphere). In view of this, it would be of interest to likewise test the formulation (either the three- or the four-parameter versions) for each of the atmospheric constituents considered by Gueymard and Kambezidis [2004, Table 5.4.2].

[36] When measured on a logarithm scale (e.g., dB), the atmospheric extinction A of radio signals from a distant extraterrestrial source is directly proportional to the air mass [Hachenberg, 1965; Hogg et al., 1981]. After relation (3), this implies that

equation image

[37] In principle, relation (19) could be useful for space geodetic techniques, such as very long baseline interferometry (VLBI) or Global Positioning System (GPS), because it relates signal delay with its own attenuation. However, its practical application comes up against serious technical problems that limit it, at present, to a theoretical scope.

Acknowledgments

[38] The help of Jesús Lago Macía in the drawing of Figures 1 and 2 is appreciated.

Ancillary