Estimating atmospheric CO2 from advanced infrared satellite radiances within an operational four-dimensional variational (4D-Var) data assimilation system: Results and validation



[1] More than a year of Atmospheric Infrared Sounder (AIRS) radiance observations have been assimilated in the European Centre for Medium-Range Weather Forecasts four-dimensional variational (4D-Var) data assimilation system to estimate tropospheric CO2. The assimilation of a set of 18 spectral channels provides a mean tropospheric mixing ratio representing a layer between ∼700 hPa and the tropopause. Analysis errors for a 5-day mean on a 6° by 6° averaging grid box are on the order of 1%. Comparisons with independent flight data from Japanese Airlines and National Oceanic and Atmospheric Administration Climate Monitoring and Diagnostics Laboratory are favorable. Differences between the averaged assimilation estimates and the onboard flask observations are generally within the 1-σ error bars. Currently, this work is being extended by introducing CO2 as a full assimilation model tracer variable, which will allow the operational monitoring of atmospheric CO2 using AIRS observations and observations from upcoming instruments.

1. Introduction

[2] A proper understanding of the global carbon cycle is critical for understanding the environmental history of our planet and its human inhabitants, and for predicting and guiding their joint future [Global Carbon Project, 2003].

[3] One path to achieve the above goal is by inferring surface fluxes of CO2 from atmospheric CO2 observations using an inverse transport model. Significant progress has been made in the last decade using observations from the surface flask networks [e.g., GLOBALVIEW-CO2, 2003] (available on the Internet via anonymous ftp to, Path: ccg/co2/GLOBALVIEW) by improving transport models and inversion techniques [e.g., Gurney et al., 2002, 2004; Peters et al., 2004; Rödenbeck et al., 2003]. However, these surface flask networks, although being extended, are limited in number and geographical area, which limits the inversion approach. Rayner and O'Brien [2001] showed that the greater coverage in time and space provided by satellite data can improve existing surface flux estimates even though the precision of individual measurements may be an order of magnitude lower than those estimated from the air sampling network.

[4] Engelen et al. [2001] performed a simulation study to look at the capabilities of the Atmospheric Infrared Sounder (AIRS), Chédin et al. [2003] did similar simulations for the Infrared Atmospheric Sounding Interferometer (IASI), and O'Brien and Rayner [2002] studied the near-infrared option, which might be realized by the Orbiting Carbon Observatory (OCO) mission. AIRS and IASI (will) observe emission in the infrared part of the spectrum and are therefore not very sensitive to lower tropospheric CO2 (below 800 hPa), but they can observe every location two times per day. OCO will observe reflection in the near-infrared and will therefore be most sensitive to lower tropospheric CO2, but can only observe CO2 during daytime. All three studies showed that the required accuracy of 2.5 ppmv for monthly mean column-integrated data on a 8° × 10° footprint [Rayner and O'Brien, 2001] is in principle achievable. Chédin et al. [2002] used real data from the Tiros Operational Vertical Sounder (TOVS) to infer atmospheric CO2 concentrations in the tropics, and Engelen et al. [2004] described the use of AIRS observations in the European Centre for Medium-Range Weather Forecasts (ECMWF) four-dimensional variational (4D-Var) data assimilation system to infer tropospheric CO2 concentrations.

[5] In this paper, we present the current status of the CO2 data assimilation system as described by Engelen et al. [2004]. In section 2 we give a brief summary of the data assimilation system and describe the main changes compared to system described by Engelen et al. [2004]. Section 3 shows monthly mean results, while section 4 describes comparisons with independent CO2 observations.

2. Short Description of the Assimilation System and Recent Changes

[6] Engelen et al. [2004] described the setup of the current CO2 data assimilation system at ECMWF. In short, radiance observations from the Atmospheric Infrared Sounder (AIRS) [Aumann et al., 2003] are operationally assimilated in the ECMWF 4D-Var data assimilation system together with many other observations to constrain the dynamics and thermodynamics of the assimilation model [McNally et al., 2005]. For the CO2 assimilation experiments, a CO2 column variable is added to the minimization state vector of the analysis system at all available AIRS observation locations within the 6 hour time window of each analysis. This means that an analysis does not provide a full three-dimensional CO2 field for each analysis cycle, as is the case for most other variables, but individual CO2 estimates for all AIRS observation locations. There is therefore no CO2 transport within the 6 hour analysis window and there are no a priori horizontal correlations applied to constrain the CO2 estimation problem. The only explicit prior CO2 constraint is in the vertical by assuming a well-mixed profile. Because the CO2 mixing ratios are estimated within the regular minimization, they fully use the information about the other meteorological variables, such as temperature and water vapor, which are constrained by various data sources (e.g., AMSU-A, AMSU-B, SSMI, radiosondes) as well as the assimilating model and background state that is produced with a 3 hour forecast from the previous analysis. This is a significant difference with stand-alone satellite retrievals that generally use only observations from the same satellite platform. For the CO2 experiments we ran the assimilation model with 60 levels at resolution T159, which is approximately 1.125° by 1.125°.

2.1. CO2 Column Variable

[7] Engelen et al. [2004] estimated two column amounts: a tropospheric and a stratospheric column separated by a variable tropopause. However, careful analysis of the results showed that there were problems with channels that had significant sensitivity both in the troposphere and the stratosphere. Because there was no correlation between the two layers applied by the background constraint, dipole structures could arise between the troposphere and stratosphere (because the temperature lapse rate changes sign). When the bulk of the sensitivity was in the stratosphere, a strong increment in the tropospheric CO2 value could be compensated by a small increment in the stratosphere. the fact that stratospheric temperatures in our assimilation model often suffer from biases increased this problem. Therefore the channel selection was changed to include only 18 channels that are mainly sensitive to tropospheric CO2 as is shown by the weighting functions in Figure 1. This also implied that the stratospheric column could no longer be estimated. Furthermore, we changed the values of the background estimate to a single global mean value of 376 ppmv. This made the interpretation of the results more straightforward compared to the previously used zonal mean monthly mean background, because any horizontal structure cannot be provided by the background. The 376 ppmv is based on the approximate annual mean of 2003 for the Mauna Loa flask station on Hawaii. Although this value is not necessarily the same as the global annual mean for 2003, it is close enough to minimize the background bias on the timescale of a year. For comparison, the variation due to the seasonal cycle is on the order of 10 ppmv.

Figure 1.

CO2 weighting functions for the 18 channels used in the assimilation. The weighting functions represent the change in brightness temperature (dBT) for a 1% change in CO2 mass mixing ratio at every level.

[8] An extra benefit of the small channel selection was that it became computationally affordable to estimate the analysis error using the full Bayesian equation instead of relying on the neural network approach as described by Engelen et al. [2004]. Individual analysis error estimates (σa) are calculated using

equation image

where σb is the background error, R is the observation error covariance containing both the measurement error covariance (O and the observation operator error covariance F matrices, and H are the CO2 Jacobians (sensitivity of the observed brightness temperatures to a change in the CO2 column concentration). The observation error covariance matrix is currently a diagonal matrix containing estimates of the combined instrument and observation operator errors with variances set to (0.6 K)2. The observation operator is a fast radiative transfer model (Radiative Transfer for the TIROS Operational Vertical Sounder (RTTOV8) [Matricardi et al., 2004]) that links the atmospheric profile variables to a simulated radiance. The error estimate therefore contains uncertainties in spectroscopy and model approximations.

[9] The analysis errors estimated using equation (1) do not take horizontal and temporal error correlations into account. However, these correlations are introduced in the assimilation system itself (e.g., through error correlations of the temperature field). We therefore previously defined an upper and lower estimate for the error of the gridded time-averaged product by assuming either full horizontal and temporal correlation between the individual error estimates or zero correlation, respectively. This wide range of possible mean error was not ideal and we therefore have tried to estimate these error correlations empirically. A statistical analysis of a large ensemble of analysis error estimates was used to infer an approximation of the error correlation. The error correlation is here defined as

equation image

and Figure 2 shows this correlation as a function of distance between the error estimates in degrees as well as of time separation in days. The figure shows that for observations of the same day the correlation falls of to zero within a radius of ∼12°. The time correlation is becomes very small within ∼5 days.

Figure 2.

Estimated error correlations as a function of distance and time separation.

[10] Table 1 shows the effect of using these estimated error correlations on the mean error estimate in the tropics for various grid box sizes and averaging time periods. While the maximum error estimate (using full error correlations) remains the same (4.6 ppmv) for all combinations, the minimum error (using zero error correlations) is clearly a function of equation image, where N is the number of observations used in the average. This lower estimate is far too optimistic as can be seen from the mean errors using the more realistic estimated error correlations. These estimates vary from 1.3 ppmv for a 10° × 10° grid box size over a 30-day period to 3.7 ppmv for a 1° × 1° grid box size over a 5-day period. These estimates fall well within the target value of 2.5 ppmv for a monthly mean as described by Rayner and O'Brien [2001]. It is important to note that these estimates only reflect the random errors. Any systematic errors that have not be corrected will deteriorate the results. Systematic errors can arise from errors in the radiative transfer, errors in the cloud detection, and systematic errors in the temperature and water vapor fields of the final analysis.

Table 1. Mean Errors as a Function of Spatial and Temporal Averaging Using Full Error Correlation (r = 1), Zero Error Correlation (r = 0), and Error Correlations as a Function of Spatial and Temporal Distancea
r = 0r = 1r = f(x, t)r = 0r = 1r = f(x, t)
  • a

    Mean errors are given in ppmv.

10° × 10°
5° × 5°
1° × 1°

2.2. Cloud Detection

[11] In the first results described by Engelen et al. [2004], high CO2 values were seen in the western Pacific region, a major area of tropical convection in February. There was no direct explanation of these increased values and it was suggested that the cloud detection could have had an effect on the CO2 estimates. The cloud detection scheme for AIRS is described by McNally and Watts [2003], and the details will not be reproduced here. In summary it is a novel technique for the identification of clear channels at a particular location rather than the more traditional approach of identifying completely clear locations. Departures of the observed spectrum from clear-sky background values are first reordered into a vertically ranked space (i.e., in order of increasing sensitivity to cloud), in which the characteristic signature of cloud becomes monotonic and more readily identifiable as shown in Figure 3. A digital filter is then applied to the departures to reduce the instrument noise (and noise due to errors in the background estimate of the atmospheric state). This essentially isolates the pure cloud signal such that the level (or channel in the ranked space) where the cloud contamination first becomes significant can be determined. Channels ranked above this level (i.e., less sensitive to cloud) are retained for assimilation and channels ranked lower (i.e., more sensitive to cloud) are discarded. The result of such an approach is that the data coverage in the analysis is different for each channel. Channels with weighting functions peaking higher in the atmosphere are used extensively whereas channels with lower peaking weighting functions (e.g., window channels) are used significantly less often. The cloud detection scheme has a number of tunable parameters, such as the window width of the digital filter and the gradient threshold of the smoothed curve that stops the detection, which have initially been set to rather stringent values. This possibly results in the wrongful rejection of some clear data, but ensures that errors due to undetected residual cloud contamination in channels passed as clear are very small (estimated to be typically less that 0.2K for a midtropospheric temperature sounding channel).

Figure 3.

Example of the cloud detection scheme. Solid circles denote channels flagged as cloudy. See text for further explanation.

[12] Although the cloud detection works very well in most cases [McNally and Watts, 2003], the assumption that there are no systematic errors proved to be a problem in the tropical convective areas. The detection of thin cirrus (thin enough to still show the atmosphere and/or surface underneath it) was compromised by large systematic errors in the background water vapor profiles that affect the lowest peaking channels in the long-wave band (sensitive to water vapor). Figure 4 (left) shows an example of an undetected thin cirrus cloud. A large error in the water vapor background profile has lifted these lowest peaking channels in the long-wave band up toward the zero departure line. The cloud detection scheme therefore detects a cloud at channel 123, which is a very low peaking channel. It fails to detect the real cloud effect that is causing the smoothed curve to drop again outside the threshold range. The simple fix that was implemented consisted of removing the 30 lowest peaking channels from the cloud detection to remove the biased water vapor effect. Figure 4(right) shows the result. Because the channels used are now not sensitive to water vapor anymore, the cloud detection is able to detect the cirrus cloud, which affects all the channels with an index higher than 79. This fix improved the cloud detection in the tropics and therefore removed some (but not all, as will be shown in section 4) anomalous CO2 estimates around the edges of convective clouds. The channels that were removed from the cloud detection and therefore flagged as cloudy for all observation locations were not used for the CO2 estimation itself. The above described cloud detection problem affected other convective areas as well, but we have focused on the Pacific, because we have independent CO2 data in that area and also because the western Pacific is the most important area for large-scale tropical convection.

Figure 4.

Example (left) of undetected thin cirrus with the old cloud detection scheme and (right) of the detection of the thin cirrus with the new cloud detection scheme.

3. Results

[13] As done by Engelen et al. [2004], monthly mean results are presented by averaging on a 1° by 1° latitude-longitude grid. Within a grid box the data were averaged using the individual analysis error estimates as weights. This 1° by 1° grid was then smoothed with a 15° by 15° moving boxcar average for clarity. Monthly mean errors were averaged on the same grid taking the estimated error correlations into account. AIRS data in the period from 1 January 2003 until 31 March 2004 have been processed, and Figure 5 shows monthly mean CO2 analysis results for March 2003, September 2003, and March 2004. Also shown is the monthly mean analysis error for March 2003. The mean errors for the other two months are very similar and therefore not shown. The largest signal in atmospheric CO2 concentrations comes from the terrestrial biosphere [e.g., Erickson et al., 1996]. A strong seasonal cycle is produced, although the annual net biosphere flux is very close to zero. The terrestrial biosphere also creates a latitudinal gradient in the atmospheric concentrations due to the large amount of land in the Northern Hemisphere compared to the Southern Hemisphere. This latitudinal gradient is amplified by anthropogenic emissions that mainly originate from the Northern Hemisphere [Denning et al., 1995]. Both the seasonal cycle and the latitudinal gradient are visible in the results of Figure 5. It is encouraging to see that the assimilation is capable of producing these spatial and temporal variations without having that information in the background, as was the case in the previous results of Engelen et al. [2004]. Another marked difference with the earlier results is that the high values in the tropical convective areas are significantly reduced and fit better within the CO2 patterns of the surrounding areas. Furthermore, March 2004 shows generally higher CO2 concentrations than March 2003, probably because of the upward trend in global atmospheric CO2. The difference between March 2003 and March 2004 at the location of Hawaii is 1.6 ppmv compared to the 2.3 ppmv observed at the Mauna Loa flask station. This is reasonably close, especially considering that the observations represent different parts of the troposphere. The monthly mean error shows the clear dependence of the analysis error on the temperature lapse rate as well as the thickness of the tropospheric layer. Errors are smallest in the tropics were the tropopause is high and the temperature lapse rate is large, while they increase at higher latitudes where the tropopause is lower. The relatively low errors over Europe are caused by a higher tropopause (deeper tropospheric layer) in the subtropical air mass. The data density does not significantly affect the monthly mean error, because of the applied error correlations (see section 2.1). Monthly mean errors range from about 1 ppmv to 6 ppmv. These are the random errors; uncorrected systematic errors are not included in the estimate. The patterns in the CO2 distribution are quite reasonable and compare well with independent CO2 simulations over ocean from the LMDz model [Sadourny and Laval, 1984; F. Chevallier, personal communication, 2005].

Figure 5.

Monthly mean analysis results for March 2003, September 2003, and March 2004 as well as the monthly mean analysis error for March 2003.

4. Validation

[14] The presentation of monthly mean results and the comparison with model simulations is interesting by itself, but an important check of the validity of our analysis results is by comparing these results to independent observations of atmospheric CO2. There are only very few data sources for 2003. We cannot use the surface flask data as our estimates represent a layer between ∼700 hPa and the tropopause, while the surface flasks are sampled in the boundary layer. Only if we are sure that the full tropospheric CO2 profile is well mixed, a comparison would be useful. However, two data sets of middle to upper tropospheric CO2 do exist and comparisons are presented here.

4.1. Comparisons With Japanese Airlines Measurements

[15] The first data set consists of CO2 data sampled from Japanese Airlines (JAL) commercial airliners flying between Australia and Japan [Matsueda et al., 2002]. These observations consist of automatic flask samples gathered at altitudes between 8 and 13 km on biweekly commercial flights. For 2003, 21 flights were available for our comparisons. Figure 6 shows the CO2 annual cycle for both the flight observations and the assimilation estimates. For the full processed period (1 January 2003 to 31 March 2004), CO2 estimates were sampled in a 6° × 6° box around the location of the flight observation over a period of 5 days around the date of the flight observation. A minimum of 100 analysis estimates for the average was required to produce a mean CO2 value for each location. This resulted in annual cycle plots for 12 different latitudes, because each JAL flight has 12 flask observations at approximately the same latitudes.

Figure 6.

Comparison of CO2 estimates with JAL observations for three different latitude zones from January 2003 to March 2004. Missing ECMWF data are caused by extensive cloud cover in the area.

[16] For clarity, we generated three plots from these 12 annual cycles, shown in Figure 6, that represent the Northern Hemisphere region, the equatorial region, and the Southern Hemisphere region, by averaging three latitude plots together for each region. The CO2 analysis estimates represent a thick layer, while flight observations are taken at a certain height level, but Sawa et al. [2004] showed that (at least) in February 2000 the CO2 mixing ratios in the zone 13°–24° N are very similar at various altitudes and agree well with the JAL observations. Figure 6 shows that the analysis estimates follow the JAL observed annual cycle quite well. All differences fall within the 1-σ error bars and are of the order of 1 ppmv in most cases. There is a clear improvement compared to the used background, which is 376 ppmv throughout the year. The main anomaly can be seen in both the Northern Hemisphere and the Southern Hemisphere in January and February. The analysis estimates are consistently higher than the JAL observations for this period. The 12 individual annual cycles show the same behavior, although the results have more random error because of less averaging.

[17] Figure 7 shows geographical comparisons for January and May 2003. The figure for 20 January 2003 is an example of a bad match between the JAL observations and the analysis results, while the figure for 20 May 2003 is an example of a good match. The main problem area in the January plot is an area affected by clouds, although this is not immediately visible in the smoothed average. There is a small area that is rightly detected as cloudy, but observations around the edges of this area, detected as cloud-free, provide high CO2 values. This causes the average CO2 field to be high compared to the surrounding area. These observations that are detected as cloud-free, still suffer from problems in the cloud detection related to the background water vapor field. In contrast, the May plot shows nice correspondence in the north-south gradient. This difference in the cloud detection between January 2003 and May 2003 is most likely caused by the effect of the GOES-9 satellite on the moisture fields of the ECMWF analysis. The normal geostationary satellite over the Pacific area was GMS-5, but continuing problems with the onboard imager required a replacement by the GOES-9 satellite in May 2003. Therefore, during the critical (in terms of amount of convection over the Pacific) months of January, February and March the geostationary constraint on the humidity field was lacking. After the middle of May, this was corrected with the GOES-9. This is a cautionary example showing the impact of seemingly unrelated satellite instrument changes on the CO2 estimates. Continuous validation of the results is therefore of great importance. Currently, efforts are under way to make the cloud detection more robust.

Figure 7.

Comparison of CO2 estimates with JAL observations for 20 January 2003 and 20 May 2003.

4.2. Comparisons With Climate Monitoring and Diagnostics Laboratory Measurements

[18] Another source of data to validate our analysis results are the vertical profiling sites managed by the National Oceanic and Atmospheric Administration Climate Monitoring and Diagnostics Laboratory (NOAA/CMDL). Air samples are collected using an Automated Air Sampling System aboard chartered aircraft at a frequency of about once a month at several sites. These samples are then analyzed at CMDL to provide concentrations of CO2, CH4, CO, H2, N2O, and SF6 as a function of altitude (CMDL Summary Report 27, 2004; available from Most sites are located in the continental United States and therefore outside the tropical area where our analysis estimates are most reliable. However, three sites are located elsewhere: Molokai Island (Hawaii; 21.23°N, 158.95°W), Rarotonga (Cook Islands; 21.25°S, 159.83°W), and Santarem (Brazil; 2.85°S, 54.95°W). Figure 8 shows 4 profiles measured at the Hawaii site for 2 April 2003, 5 June 2003, 17 August 2003, and 9 October 2003. The solid vertical line represents the CO2 analysis estimate with the 1-σ error bar shown as the horizontal line. The vertical dashed line is the background value used in the analysis. Only four profiles are shown for clarity, but they were chosen to be representative of the other available profiles in 2003. The CO2 analysis values were calculated by averaging within a 6° × 6° box of the CMDL site and over an period of 5 days around the date of the CMDL observation. The average error was calculated the same way as for the comparisons with the Japanese data. The figure shows that the analysis estimates are closer to the CMDL observed profiles than the background and fit well within the 1-σ error. The changes due to the seasonal cycle are clearly captured. Figure 9 shows scatter diagrams with all available data for Hawaii, Rarotonga, Santarem, and Harvard Forest. Each plot also provides the linear correlation coefficient (r). The horizontal dashed line represents the background value used in the assimilation. The analysis values correlate well with the CMDL data, although the scatter is not negligible. This means that the annual cycle is generally well captured as with the Japanese data, but individual comparisons can have differences up to 3 ppmv. The error bars seem large in this kind of scatter diagram, but are well within 1%. The Harvard Forest plot was added to show an example for higher latitudes.

Figure 8.

ECMWF tropospheric mean mixing ratio (solid lines) compared to CMDL flight profiles (solid circles) for 2 April, 5 June, 17 August, and 9 October 2003 at Molokai Island, Hawaii. The analysis background value is represented by the dashed lines, and the mean analysis error is represented by the horizontal solid line.

Figure 9.

Scatter diagrams of analysis CO2 estimates versus CMDL flight data for Hawaii, Cook Islands, Brazil, and; Harvard Forest.

5. Summary

[19] More than a year of AIRS data have been assimilated in a CO2 configuration of the ECMWF 4D-Var data assimilation system. Eighteen spectral channels sensitive to tropospheric CO2 are used to estimate the mean CO2 concentration in layer between 700 hPa and the tropopause. Results look realistic and compare well with model simulations and independent CO2 observations.

[20] The analysis estimates are currently not constrained by the transport model and the background constraint is very weak. Therefore individual estimates are quite noisy and require spatial and temporal averaging. However, 5-day averages on a 6° by 6° grid compare well with observations from the JAL commercial airliner flying between Japan and Australia at altitude of ∼11 km. The estimated mean errors are of the order of 1%, which should be sufficient to have an impact on current flux inversions. Comparisons with profile data from the CMDL network shows similar agreement.

[21] An important problem area of the assimilation estimates is undetected systematic errors. Not only can biased temperature and water vapor fields directly bias the CO2 estimates through the radiative transfer modeling, but it was also shown that undetected clouds can have an impact on the estimated CO2 values and the cloud detection algorithm was adjusted to minimize this problem. Continuing validation of the analysis results is required to remove any remaining systematic errors.

[22] On the basis of the results presented in this paper we will introduce CO2 as a tracer in the ECMWF transport model, which will make it part of the full 4D-Var assimilation, including the constraint in the time dimension provided by the transport model. This will require a more sophisticated background constraint that realistically represents the background errors.

[23] At the same time first efforts will be made to use the CO2 estimates in a flux inversion model. This will show where data from the flask network and the data presented in this paper agree and disagree. It will also provide an estimate of the added information from the satellite estimates compared to just using flask observations.


[24] The work described in this paper was financially supported by the EC project COCO (EVG1-CT-2001-00056). The authors are very grateful to Hidekazu Matsueda from the Meteorological Research Institute in Japan for providing the JAL flight data. The authors are also very grateful to Pieter Tans from NOAA/CMDL for providing the CMDL flight data. Furthermore, we would like to thank two anonymous reviewers for their very useful comments on the manuscript.