• Open Access

Advances in simultaneous atmospheric profile and cloud parameter regression based retrieval from high-spectral resolution radiance measurements



[1] The dual-regression (DR) method retrieves information about the Earth surface and vertical atmospheric conditions from measurements made by any high-spectral resolution infrared sounder in space. The retrieved information includes temperature and atmospheric gases (such as water vapor, ozone, and carbon species) as well as surface and cloud top parameters. The algorithm was designed to produce a high-quality product with low latency and has been demonstrated to yield accurate results in real-time environments. The speed of the retrieval is achieved through linear regression, while accuracy is achieved through a series of classification schemes and decision-making steps. These steps are necessary to account for the nonlinearity of hyperspectral retrievals. In this work, we detail the key steps that have been developed in the DR method to advance accuracy in the retrieval of nonlinear parameters, specifically cloud top pressure. The steps and their impact on retrieval results are discussed in-depth and illustrated through relevant case studies. In addition to discussing and demonstrating advances made in addressing nonlinearity in a linear geophysical retrieval method, advances toward multi-instrument geophysical analysis by applying the DR to three different operational sounders in polar orbit are also noted. For any area on the globe, the DR method achieves consistent accuracy and precision, making it potentially very valuable to both the meteorological and environmental user communities.

1 Introduction

[2] Hyperspectral measurements have a high vertical resolving power that enables precise spectral and radiometric calibration [Tobin et al., 2006], as well as precise inversion of the radiative transfer equation [Smith et al., 2009]. The resulting retrieved profiles have improved accuracy at high vertical resolution with a demonstrated capability of capturing mesoscale details of the atmospheric state to the benefit of NWP (numerical weather prediction) assimilation and climate research studies [Chahine et al., 2006]. European Centre for Medium-Range Weather Forecasts studies confirmed that hyperspectral sounder retrievals, e.g., from AIRS (Atmospheric Infrared Sounder) and IASI (Infrared Atmospheric Sounding Interferometer), are vital components for NWP and improve forecast skill and accuracy [Thepaut et al., 2011]. Furthermore, recent efforts were made to expand assimilation to cloud-affected radiances at high-spectral resolution in order to provide better analyses and forecasts in cloudy regions [Bauer et al., 2011]. Although not yet operationally implemented, retrieval assimilation has been studied [e.g., Rodgers, 2000] as well, and the use of retrieved temperature and moisture data (in addition to model-analysis fields) promises to improve prediction skill and efficiency of the assimilation [Migliorini, 2012]. For the assimilation of soundings into NWP models, regional bias differences between the retrieval and the model fields can be removed in order to alleviate the influence of a statistical bias in the assimilation process. Hence, accurate retrievals from soundings under both clear- and cloudy-sky conditions are becoming indispensible sources of mesoscale and global-scale information in a wide range of applications.

[3] Accurate atmospheric profile, surface, and cloud parameter retrievals from cloud-contaminated radiances can be achieved, as has been demonstrated in several published studies [e.g., Smith, 1967; Susskind et al., 2003; Smith et al., 2004, 2005; Zhou et al., 2007; Weisz et al., 2007b; Zhou et al., 2009; Liu et al., 2009]. In part, this achievement is due to advances made in cloudy-sky radiative transfer models for the infrared spectrum [Yang et al., 2001, 2003; Wei et al., 2004].

[4] The dual-regression (DR) method retrieves atmospheric parameters in real time from polar-orbiting high-spectral resolution instruments like AIRS on Aqua, IASI on MetOp-A and MetOp-B, and CrIS (Cross-Track Infrared Sounder) on Suomi-NPP (National Polar-orbiting Partnership) regardless of surface and/or cloud condition. A general overview of the DR method is given by Weisz et al. [2011] and Smith et al. [2012]. The authors successfully demonstrate that this technique provides reliable mesoscale atmospheric temperature and moisture variation, and is capable of detecting stability changes occurring before severe weather events. This successful demonstration, and the fact that the method is computationally efficient, indicates the value of the retrieval products for severe weather forecasting applications, specifically since it is anticipated to lead to improved hurricane intensity and track forecasts [Li and Liu, 2009].

[5] With the DR method, geophysical parameter retrievals are obtained at single field of view (FOV) resolution using two sets of eigenvector regression relations; one trained on clear-sky atmospheric profile conditions and the other trained on cloud height stratified cloudy atmospheric profile conditions. The DR method relies on the basic concept that under clear conditions, the clear- and cloudy-trained retrieved temperature profiles will be very close, whereas under cloudy conditions, the clear-trained solution will be colder than the cloud-trained solution below the cloud top. The cloud heights are specified as that level where the profiles start to deviate from each other. The profiles are then combined to create the final sounding product. The key to an accurate profile determination is the correct specification of the height of the highest cloud within the instrument's field of view. The cloud height is used to assign a cloud height class to the FOV that in turn determines the set of regression coefficients used in the retrieval.

[6] The DR method is implemented as a stand-alone algorithm that converts calibrated radiances from any of the three polar-orbiting sounders, AIRS, IASI, and CrIS, into Level 2 (L2) products. It has been released as part of the CIMSS (Cooperative Institute of Meteorological Satellite Studies) CSPP (Community Satellite Processing Package) software package and is open-source in the public domain. The method is unique in its multi-instrument scope and retrieval capability with which the Earth surface, atmosphere, and clouds can be characterized anywhere on the globe twice daily (corresponding to satellite overpasses). Moreover, the regression relations are defined by training data sets that capture the variance of atmospheric parameters and comprise global coverage over all four seasons, which lends it consistent accuracy and precision irrespective of the geolocation and time of the measurements. Although model data is available globally as well, it strongly depends on in-situ measurements for depiction of mesoscale features, which renders it reliable only in those areas with established in-situ measurement networks (e.g., North America and Europe), and less reliable in polar and oceanic regions, as well as large parts of Asia and Africa. The DR method's reliable geophysical characterization is due to the fact that information is retrieved from radiance measurements alone, the quality of which is consistent across the globe and independent of geographic area.

[7] In this paper, we revisit the simple linear regression basis of the DR method but elaborate specifically on those steps that are taken toward retrieving nonlinear parameters, e.g., cloud top pressure (CTP). These physically based decision steps greatly enhance the accuracy with which geophysical parameters can be characterized at instrument resolution.

[8] An overview of the core statistical nature of the DR method is given in section 2. An in-depth discussion of the geophysical steps taken to advance the nonlinear capabilities of the DR method follows in section 3 with a focus on the correct determination of the cloud top pressure. The criteria used to determine whether a FOV is cloudy or clear are detailed in section 4. Retrieved cloud top pressures are converted and compared with cloud top heights (CTHs) from CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization), the lidar instrument on board CALIPSO (Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation) [Winker et al., 2009], in section 5. In particular, for thin cirrus and other tenuous clouds, quantifying the accuracy of the cloud height retrievals using measurements from a lidar instrument like CALIOP is invaluable. Section 5 also describes how sounder radiances and retrieval products provide quantitative information about the atmospheric state in comparison with visible imagery from MODIS (Moderate Resolution Imaging Spectroradiometer) and VIIRS (Visible Infrared Imaging Radiometer Suite). When reasonable cloud heights have been found, the final sounding profiles are constructed from the clear and the cloudy regression solutions. This process is described in section 6. A summary of the main conclusions and future research is given in section 7.

2 Algorithm Outline

[9] The mathematical basis of the retrieval method described here is linear regression, which has been applied to the inversion of satellite radiance measurements for decades [e.g., Smith et al., 1970; Smith and Woolf, 1976; Huang and Antonelli, 2001; Zhou et al., 2007; Weisz et al., 2007a]. The dual-regression (DR) method, however, makes advances toward addressing known problems associated with the nonlinearity of the inversion problem, such as cloud parameter retrievals. An overview of the DR framework is given by Smith et al. [2012]. In this section, we elaborate in detail on the decision steps that largely determine retrieval accuracy and uncertainty.

[10] Unlike the optimal estimation (or physical) retrieval scheme, regression does not rely on a location-specific a priori estimate of the atmosphere to invert sounder measurements. Instead, it relies only on the variance and mean of the atmospheric state. The variance and mean are calculated once from an ensemble of all possible values (temporally and spatially) for each parameter making up the atmospheric state. These statistical values are then used for inverting sounder measurements anywhere on the globe. The term “dual-regression” refers to the fact that two sets of regression coefficients are computed from ensembles of data (or training sets), one for clear and one for cloudy atmospheric states, respectively. Each set of coefficients is separately applied to a radiance measurement (i.e., one radiance spectrum) to retrieve a clear and cloudy solution. The solution vector includes temperature and humidity profiles, trace gas concentrations, as well as surface and cloud parameters (height, temperature, and optical thickness). The clear solution assumes clear-sky conditions and describes the atmospheric state from the surface to top-of-atmosphere (TOA). The cloudy solution describes the atmospheric state from the cloud top to TOA, although under thin and/or broken clouds levels below the clouds are included. The cloud height is determined by a regression approach where eight overlapping cloud height categories (from 100 to 900 hPa with a 200 hPa range each) are employed. Starting from an overall cloud class (100–900 hPa), the optimal cloud class is found by iterating through the cloud classes, and the retrieval corresponding to this cloud class is retained. The cloud top estimate associated with the optimal cloud class is referred to as the regression-predicted cloud top. Classification by cloud height reduces the nonlinearity of the relationship between radiances and clouds and between radiances and humidity [Smith et al., 2005]. Prior to starting the retrieval, the co-located NCEP (National Centers for Environmental Prediction) GDAS (Global Data Assimilation System) temperature profile is used to define the tropopause level and the existence of a temperature inversion.

[11] A DR retrieval advances through the following steps:

  1. [12] The temperature profiles from both clear and cloudy solutions together with a model-analysis (i.e., NCEP GDAS) temperature profile are paired as follows: clear versus cloudy, clear versus model, and cloudy versus model. Cloud top estimates are assigned to the lowest level of intersection for each pair. The highest possible cloud top altitude is selected (and its cloud class is determined) from all three pairs of temperature profiles as well as the regression-predicted cloud top estimate. This process is described in section 3.

  2. [13] If the cloud class determined in step 1 does not match the input cloud class from the regression-predicted cloud top, a new cloudy solution of the atmospheric state is retrieved, this time using as input the cloud class determined in step 1.

  3. [14] If necessary, steps 1–2 are repeated until the cloud top determined in step 2 is consistent with the current input cloud class to the cloudy retrieval. This process ensures that the atmospheric state of the cloudy solution is in agreement with the determined cloud class. If convergence does not occur within four iterations, the retrieval is flagged as unsuccessful.

  4. [15] A cloudy solution is retrieved for all FOVs, even in the event of clear-sky conditions. It is discarded, however, when the FOV fails the following two clear-sky criteria: (i) a criterion based on the difference between the clear solution surface air temperature and the model surface air temperature and (ii) a criterion based on the effective cloud emissivity as defined in section 4 below. If both of these values are smaller than a predefined threshold, then the FOV is considered to represent clear-sky conditions, and the cloudy solution is discarded.

  5. [16] Based on the clear-sky decision made in step 4, the final profile is created from the clear and/or cloudy solution as detailed in section 6 below.

[17] Three types of quality flags are currently assigned to the final solution to describe (i) retrieval success, (ii) uncertainty, and (iii) accuracy. A retrieval is flagged as unsuccessful either due to bad radiance input or failure to converge on a cloud height class (see step 3). Retrieval uncertainty, in turn, is determined by the certainty associated with the clear-sky test (step 4) as described in section 4 below. Finally, accuracy is determined by comparing the retrieval solution with model-analysis temperature profiles.

[18] A distinguishing characteristic of the DR method is that the cloud top is derived from vertical geophysical data (temperature profiles), as opposed to being retrieved only from a few selected spectral radiance measurements.

[19] Although emphasis in this paper is placed on cloud top retrieval, any geophysical parameter can be retrieved with this method as long as it is well represented in the training data sets and the parameter is radiatively active in the infrared spectrum. The retrieval (or L2) products of the current version of the DR software include temperature, moisture and ozone profiles, surface skin temperature, surface emissivity (at full spectrum), carbon dioxide concentration, cloud top pressure, and cloud optical thickness. Dew point temperature and relative humidity profiles, total precipitable water, total ozone amount, lifted index, convective available potential energy, cloud top temperature, and effective cloud emissivity are computed from the retrieved temperature and moisture profiles.

3 Cloud Top Determination

[20] This section illustrates the cloud top determination process (see previous section, step 1) from temperature profile comparisons of the clear and cloudy regression solutions, as well as model-analysis data (NCEP GDAS). For each pair of profiles (clear versus cloudy, clear versus model, and cloudy versus model), the level of intersection is found by iterating through the atmospheric levels starting at the tropopause. First, a threshold level is defined where the temperature difference between the pair of profiles exceeds a predefined value at all the levels below. A value of 3 K is used to avoid selecting an erroneous level due only to retrieval noise. From this threshold level and the temperature at that level, the actual cloud top (i.e., where the profile temperature difference is 0 K) can be estimated assuming a nominal atmospheric temperature lapse rate of −0.5 K/100 m. The cloud tops determined from each profile pair along with the regression-predicted cloud top are compared, and the highest level is saved as the final result.

[21] This simple comparison has proven to be robust and effective. It is based on the principle that if clouds are present, then the clear regression solution will be colder than the cloudy solution below the cloud. In turn, the cloudy solution can be expected to closely resemble prevailing geophysical conditions if it was retrieved using the regression coefficients corresponding to the correct cloud class.

[22] If the temperature profile difference of the clear and cloudy solutions is well defined (the magnitude of this difference depends on the height and optical thickness of the cloud), then the cloud top determination is straightforward, as is shown in Figures 1a–1c. For the FOV depicted in Figure 1a, the profiles start to deviate from each other at around 190 hPa; this level can be considered as the cloud top. Figures 1b and 1c illustrate similar cases with lower cloud tops, which are located at 389 and 699 hPa, respectively. In comparison with co-located CALIOP retrievals, the clouds in all these cases have optical depth (OPD) values exceeding 0.5, which makes accurate cloud detection possible. In contrast, the examples shown in Figures 1d–1f have small OPD values, which make cloud determination more challenging. If no level is found, where the profiles show clear differences, then the cloud top is temporarily assigned to the surface, and the profile is likely to be clear (e.g., Figure 1d). For the FOV depicted in Figure 1e, a low cloud (at 755 hPa) is found, but a decision on whether the profile is actually clear or cloudy has yet to be made. This process will be discussed in the next section. When a temperature inversion is present at the surface (Figure 1f), as is often the case in Polar Regions, the cloud top determination becomes more complex. Rather than picking the lowest level of all the intersections (between the profiles of a pair), selecting the highest intersection gives more reasonable results in the presence of surface inversions. For this case (Figure 1f), a cloud top is assigned to 190 hPa.

Figure 1.

(a–f) Selected clear, cloudy and model (GDAS) profiles to illustrate DR cloud top determination. CALIOP cloud top pressures (in hPa) and optical depths are stated in the title of each panel.

4 Decision on Clear/Cloudy

[23] Once a cloud top has been defined, another test is applied to check whether the profile is really cloudy or not (section 2, step 4). Two criteria are currently used to test if a field of view (FOV) is clear or cloudy.

[24] The first criterion is based on the difference between clear retrieved and model surface air temperatures. The latter is defined as the value of the temperature profile at the surface level. A robust value for this threshold that yields reliable results was empirically determined to be 2.5 K. This value accounts for both model and retrieval uncertainty and has proven to reflect real geophysical differences.

[25] The second criterion is based on the effective cloud emissivity. It is a ratio of the difference between the clear regression surface skin temperature (ts) and model surface air temperature (ms), and the difference between the clear regression temperature at cloud top level (tc) and ms. Hence, the effective cloud emissivity e is written as e = (ts − ms)/(tc − ms). The effective cloud emissivity is constrained to values between 0 and 1. A robust value for this threshold was determined to be 0.08.

[26] For a FOV to be considered cloudy (clear), both the surface air temperature difference and effective cloud emissivity have to exceed (fall short of) the relevant thresholds. These thresholds have been determined based on a large number of global tests in comparison with space-borne active instruments (like CALIOP). In other words, these values were derived as those yielding reasonable results for most geophysical conditions on a global scale.

[27] Based on the reasoning we have outlined thus far, the FOVs depicted in Figures 1a–1c are cloudy. However, the FOVs shown in Figures 1d–1f exhibit low optical thickness and are therefore more difficult to classify as either cloudy or clear. This result is due to their low thermal contrast to clear skies. The retrieved profiles in Figure 1d are almost identical and the logic employed in the DR method classifies it as a clear case. CALIOP, on the other hand, defines this scene as an optically very thin (OPD = 0.01) high cirrus cloud. Figure 1e shows a similar case, but with some differences between clear and model profiles. The DR method classifies it as a low cloud (755 hPa), which is close to the CALIOP measurement. These cases illustrate the sensitive nature of these threshold tests and how misclassifications can happen in certain conditions. For example, high thin cirrus with low OPD can easily be misidentified as low clouds or even as clear sky; clear scenes can similarly be misidentified as thin high or very low clouds. Very low clouds are in general difficult to retrieve (for any instrument) due to insufficient thermal contrast with the Earth surface.

[28] The thresholds used in deciding whether a FOV should be considered clear or cloudy, may not hold in every cloud scene under all geophysical conditions. Hence, a quality flag was developed to describe the uncertainty associated with the clear/cloudy decision. The quality flag is based on the logic that if the values of the surface air temperature difference and effective cloud emissivity are very close to the stated thresholds, respectively, then the clear/cloudy decision should be regarded as uncertain. It is anticipated that this flag will aid in the interpretation of results and the identification of areas where the thresholds repeatedly yield inaccurate results and thus require a readjustment.

5 CTH Comparison With CALIOP

[29] Since much of the DR retrieval accuracy is determined by the correct cloud top determination and clear/cloudy classification, we evaluate this decision scheme by examining a large region of CALIOP/AIRS cloud top height (CTH) comparisons. The DR method provides cloud top pressure (CTP) retrievals (in hectopascals) which are converted to cloud top heights (in kilometers) by using NCEP GDAS temperature and moisture profiles. Aqua and CALIPSO are part of the same constellation of satellites, the so-called A-Train, with Aqua leading CALIPSO by ~75 s. This arrangement provides a large set of nearly simultaneous cloud observations to assist sounder cloud retrieval evaluation using CALIOP/AIRS comparisons [Weisz et al., 2007c; Kahn et al., 2008; Weisz et al., 2012].

[30] But despite simultaneous measurements and improved collocation technologies [Nagle and Holz, 2009], exact spatial correspondence between cloud products from the active lidar with those from a passive instrument is not possible. The CALIOP and AIRS do not sense the same cloud scene due to differences in spatial resolution (1–5 km versus ~14 km) and spectral operating ranges (optical versus infrared). A sounder FOV of ~14 km can contain a variety of clouds and clear scenes. For such nonuniform scenes, the differences between lidar and IR sounder L2 products can be large. For example, the sounder, due to its large FOV size, will miss clouds of small horizontal extent. And since active and passive instruments utilize different wavelengths for their measurements, their sensitivities to cloud particles differ. CALIOP is able to detect thin cirrus and tenuous cloud tops, whereas it is difficult to sense optically thin cloud with infrared instruments. For example, it is common for semitransparent cirrus clouds to exist above optically thick cumulus or stratus clouds. In these situations, algorithms using measurements from an IR instrument will generally set the cloud top close to the top of the more opaque lower cloud, hence underestimating the actual cloud height [Holz et al., 2008]. However, using the full spectrum of radiance data, including the temperature and water vapor sounding sensitive spectral channels, alleviates this ambiguity in the derived cloud top height; the sounding channel radiances provide a measure of the cloud height independent of their optical thickness [Smith and Platt, 1978].

[31] The AIRS brightness temperature (BT) for a window channel at wave number 911 cm−1 and the AIRS retrieved cloud top pressure for a swath from −65° to 65° latitude (comprising seven AIRS granules) on 18 April 2012 are shown in Figure 2. The CTP retrievals as well as the clear-sky detection (e.g. over the U.S. states of Florida and Georgia) are consistent with what can be seen from MODIS visible imagery (Figure 2, right).

Figure 2.

(left) AIRS window BT and (middle) retrieved cloud top pressure for 18 April 2012. CALIPSO's track is outlined in black. Aqua MODIS true color (band 1-4-3) image composite is shown on the right (measurement time 18:00 to 18:35 UTC).

[32] Figure 3 shows CALIOP (5 km product) and AIRS retrieved cloud heights along the track shown in Figure 2. Clouds below 1 km are not shown. In the top panel, CALIOP's total attenuated backscatter is shown in the background. In the bottom panel, CALIOP's cloud optical depths (OPD) are displayed.

Figure 3.

Cross section along the CALIPSO track for AIRS granules 180–186 (18 April 2012). (top) Shown in the background is the CALIOP 532 nm total attenuated backscatter per km per steradian. CTHs from CALIOP and AIRS DR retrieval are plotted as blue dots and red plus signs, respectively. (bottom) CALIOP optical depths.

[33] The agreement between the passive sounder and the active lidar instrument is very good. This agreement indicates that the DR method is capable of using the full infrared spectrum to correctly detect the clouds and determine their altitudes by differentiating between the molecular radiance signal from the cloudy radiance signal contained within the observed radiance spectrum. However, as stated previously, problems remain in regions of optically thin clouds (i.e., clouds with optical depths < 0.1). These clouds, often cloud edges, do not affect the infrared radiance spectrum and therefore cannot be retrieved by the DR method. An example of this deficiency is shown in Figure 3 near 18° north latitude. Furthermore, since the methodology utilizes the retrieved temperature profiles from the tropopause downward for cloud top determination, cloud tops above the tropopause are set to tropopause level. Thus, tropical clouds with overshooting tops, as shown in Figure 3 near 3° north latitude, are underestimated.

[34] As mentioned above, the DR method can be applied to any hyperspectral infrared sounder. Suomi-NPP is in the same orbital plane (afternoon orbit) as the A-train, but its slightly higher altitude results in a slightly different orbital track, although simultaneous overpasses occur every 2.7 days. For the date shown in Figures 4 and 5, namely 29 October 2012, when superstorm Sandy made landfall along the U.S. coastline, Aqua trailed NPP by approximately 45 minutes.

Figure 4.

(top) CrIS window BT, retrieved cloud top pressure (17:19–17:43 UTC) and VIIRS true color image. (bottom) AIRS window BT, retrieved cloud top pressure (18:05–18:30 UTC) and MODIS visible (true color) and infrared images. For 29 October 2012, CALIPSO's track is outlined in black.

Figure 5.

Cross section along the CALIPSO track for 29 October 2012. (top) Shown in the background is the CALIOP 532 nm total attenuated backscatter per km per steradian. CTHs from CALIOP, AIRS DR retrieval, and CrIS DR retrieval are plotted as blue dots, red and green plus signs, respectively. (bottom) CALIOP optical depths.

[35] A CrIS brightness temperature, a retrieved CrIS CTP, and a Suomi-NPP VIIRS true color image (focusing on the northern part of the region) are shown in the top panels of Figure 4. An AIRS brightness temperature, a retrieved AIRS CTP and an Aqua MODIS true color image are shown in the bottom panels of Figure 4.

[36] High-spatial resolution imagers like MODIS and VIIRS provide precise spatial details of clouds and surface characteristics. Although beneficial in validation studies, it is difficult to distinguish between low, middle, and high clouds from visible imagery. For example, the clouds northeast of the storm center can be interpreted as optically thick clouds from the VIIRS image, but they cannot be distinguished from the high thick clouds surrounding them. However, from the image showing CrIS CTP retrievals (middle top panel), it is evident that these are low clouds, which is in agreement with the 11 µm (910 cm−1) brightness temperatures (left panels). A similar situation is found in the AIRS/MODIS images of Figure 4 (bottom panels). The low clouds, which both AIRS and CrIS detect northeast of the storm center, are now confirmed by the MODIS infrared (band 31) image, where lighter colors correspond to higher brightness temperatures (i.e., lower clouds). Hence, sounder radiances and retrievals (e.g., cloud top pressure and cloud optical thickness) add quantitative information and therefore complement visible or infrared imagery from broadband imagers, which allow a more subjective interpretation only.

[37] A comparison of AIRS and CrIS retrieved cloud top heights with CALIOP measurements for this region is shown in Figure 5. Note that the temporal difference between CALIPSO and NPP is 45 minutes larger than that between CALIPSO and Aqua, possibly leading to the slight systematic altitude difference observed over the storm system between 35° and 55° north latitude. It is interesting to note these differences between AIRS and CrIS cloud height retrievals can only be observed within the storm system, but not in the areas outside the region of the storm's influence (i.e., south of 35° latitude). This noticeable difference is due to the quick changes in cloud heights in convective storms. For the current case, the 45-minute increase in the cloud heights associated with Sandy are indicative of the intensifying condition of this storm prior to landfall. Consequently, it can be inferred that a joint investigation of retrievals from two or more sounders provides useful information on convective instability, moisture transport, and atmospheric motion. The overall agreement with CALIOP in the cloud heights retrieved from both sounders is remarkable. In particular, the maritime low-level cumulus clouds between 20° and 35° latitude are retrieved correctly, according to the CALIOP measurements. Some thin clouds are missed or underestimated (e.g., 6° latitude) due to insufficient optical thickness (e.g., at cloud edges), whereas thicker clouds (e.g., around 47° latitude) are determined correctly.

6 The Final Profiles

[38] Once a cloud top has been found or the FOV has been identified as clear, the final sounding profiles are constructed (section 2, step 5). For a clear-sky FOV, the final profiles (temperature, humidity, and ozone) are assigned the clear regression solution (i.e., the cloudy solution is discarded). For a cloudy FOV, the levels above the cloud height are assigned the cloudy/clear solution for cloud heights higher/lower than 300 hPa. For the levels below the cloud, either the clear or cloudy solution is assigned depending on the magnitude of the difference between retrieved and model temperature profiles. A threshold of 3 K is employed to distinguish this assignment, i.e., if the difference between the retrieved and model profile is below 3 K, then that solution (whether it is clear or cloudy) is accepted to characterize the atmosphere below the cloud. Consequently, in many cases, no values are assigned to the atmosphere below the clouds because neither of the retrieved solutions passed the threshold test. In such cases, the final retrieval has profile data from cloud top to TOA only. The practical implementation of these decision steps is illustrated by means of three examples shown in Figure 6. The first FOV is identified as clear, and the final profile (in red) is identical to the clear regression profile. For the remaining two cases (Figures 6b and 6c), the cloud top was found to be around 200 hPa. For the second FOV, both clear and cloudy regressions start to diverge from the model temperature profile at around 400 hPa. Therefore, the final profile is saved from TOA to that level. The cloudy regression for the third FOV stays close to the model throughout the atmosphere; therefore, the entire cloudy regression retrieval is saved as the final product.

Figure 6.

(a–c) Selected temperature and relative humidity profiles to illustrate the final profile retrieval composition. NCEP GDAS, clear, cloudy, and final profiles are shown as black, dark blue, light blue, and red curves, respectively.

[39] Figure 7a shows the statistics for GDAS minus temperature and relative humidity retrievals for all FOVs (clear and cloudy), which are closest to radiosonde stations from the 20 April 2012 north-to-south swath. From the approximately 5000 profiles included in this comparison, ~3600 have been identified as cloudy. Since levels below clouds are only saved under certain conditions (e.g., thin or broken clouds), the yield near the surface is smaller with a total of ~1000 profiles. Systematic errors between GDAS and the satellite retrievals as large as 1 K and 10% in temperature and relative humidity, respectively, can be observed. Such regional statistical biases should be eliminated prior to assimilation of these retrievals into an NWP model. Figure 7b shows one selected profile compared with GDAS and a measurement from a balloon sonde, which was launched as part of a Hampton University satellite validation field campaign in 2012.

Figure 7.

(a) NCEP GDAS minus retrieval profile bias (dashed) and standard deviation (solid); (b) comparison of a retrieved profile (black) with NCEP GDAS (dashed gray) and a balloon sonde measurement (solid gray).

[40] IASI/MetOp-A retrievals are compared to GDAS products in Figure 8. IASI is a key payload element in the European MetOp series, occupying the morning orbit. MetOp-A has been in orbit since 2006, and MetOp-B was launched in 2012. We are demonstrating the capability of the DR method with IASI measurements in this paper to make clear that a seamless transition from IASI/MetOp-A to IASI/MetOp-B measurements will be possible as soon as they become available in early 2013. In addition, IASI is key in our future work, as we will focus on enhancing the trace gas capability of the DR method.

Figure 8.

(top) IASI retrieved temperature and GDAS temperature at ~700 hPa, and temperature profiles at one selected FOV. (bottom) Same as top but for relative humidity for 20 April 2012.

7 Summary

[41] NWP assimilation systems and forecast applications require accurate estimates of atmospheric profiles, surface, and cloud conditions. Measured radiances from hyperspectral instruments contain high information content on the full atmospheric and Earth surface state, but the optimal use of thousands of spectral channels can be computationally challenging in an operational (real-time) environment. Linear regression is a well-established inversion technique that is computationally far more efficient than traditional physical inversion techniques (e.g., optimal estimation) and has a demonstrated capability to obtain reasonable estimates of cloud parameters and sounding profiles across the globe. The dual-regression (DR) retrieval method is based on linear regression principles, which makes it ideal for real-time applications, but through cloud top pressure classification and a series of geophysical-based decision steps, it offers superior reliability and accuracy over a basic linear regression inversion. The DR method employs a simple concept of retrieving profiles from clear and cloudy radiance spectra via the combined use of clear and cloudy regression solutions. Nonlinearity between radiances and humidity and clouds are accounted for by the use of cloud height classification. This retrieval approach eliminates the need for a cloud mask, for cloud-clearing schemes, and, in many cases, the need for a more time-consuming physical inversion scheme.

[42] The DR method is implemented as a stand-alone algorithm that generates Level 2 retrieval products of the Earth surface, atmosphere, and clouds from calibrated radiances of any of the three polar-orbiting sounders, AIRS, IASI, and CrIS. It has been released as part of the CSPP software package and is freely available to the science community (http://cimss.ssec.wisc.edu/cspp/). It is unique in its multi-instrument scope and accuracy of retrieval products in a real-time environment. This improved accuracy is due to the fact that information is retrieved from radiance measurements alone, the quality of which is consistent across the globe and independent of regional variation in in-situ measurements.

[43] In this paper, clear and cloudy regression solutions are used in a physically based manner to determine cloud top pressure, specifically. AIRS and CrIS cloud top pressure retrievals (converted to cloud heights) compare very well with cloud heights from the active lidar instrument, CALIOP, on the CALIPSO satellite. Any discrepancies are explained by differences in instrument spatial resolution and spectral response. Differences between lidar and sounder cloud heights are most prevalent in scenes with optically thin clouds, very low clouds, or clouds of small horizontal extent. Once the cloud height is found or the FOV has been identified as clear, the final sounding profiles are constructed from the cloudy and/or clear regression solutions. The final profiles (under both clear and cloudy conditions) correspond well to model profiles within expected error bounds.

[44] Current and future work focuses on the multi-instrument capability of the DR algorithm in real-time direct broadcast applications that combine AIRS, CrIS, and IASI measurements. Furthermore, the use of near simultaneous AIRS on the Aqua satellite and CrIS on the Suomi-NPP satellite as well as IASI on the MetOp-A and MetOp-B satellites will be used to study convective instability tendencies, moisture transport, atmospheric motions, and wind. Incorporation of trace gases (e.g., carbon monoxide and methane) as additional retrieval parameters is also being investigated. Changes in the training set assembly and associated radiance simulation setup will therefore be required.

[45] The DR retrieval method can be employed with measurements from hyperspectral aircraft sounding instruments as well. It has been successfully applied to Scanning High-resolution Interferometer Sounder radiances to produce real-time moisture and temperature profiles while flying through tropical storms during NASA's Hurricane and Severe Storm Sentinel mission in 2012.

[46] Finally, another eminent task for expanding the capability of the DR software includes the use of spectral radiance measurements from microwave instruments to improve the yield of soundings below clouds. Also, the use of the L2 products in NWP and weather forecast applications will be investigated. These final steps will provide for the ultimate positive impact on weather, climate, and chemistry forecasts enabled by the global use of hyperspectral sounding data.


[47] The development of the DR processing system and its official release under CSPP was made possible through the financial support of NASA and NOAA JPSS (Joint Polar Satellite System). We acknowledge the use of MODIS data imagery from the Land Atmosphere Near-real time Capability for EOS (LANCE) system operated by the NASA/GSFC/Earth Science Data and Information System (ESDIS) with funding provided by NASA/HQ (http://earthdata.nasa.gov/data/nrt-data/rapid-response/). CALIOP data have been obtained from the Atmospheric Sciences Data Center (ASDC) at NASA Langley Research Center. VIIRS imagery has been provided courtesy of University of Wisconsin-Madison Space Science and Engineering Center.