Performance measurement with advanced diagnostic tools of all-sky microwave imager radiances in 4D-Var



In this paper, a comprehensive assessment of the impact of all-sky radiance observations in the operational European Centre for Medium-Range Weather Forecasts' assimilation and forecast system is presented using advanced diagnostic tools. In particular, the observation influence in the assimilation process and the related contribution to the short-range forecast error of radiance observations from microwave sensors is evaluated with recently developed diagnostic tools based on the adjoint version of the model. Recent operational changes to the assimilation of these observations result in a more beneficial impact on the initial condition and the short-range forecast. The largest information content is obtained for observations that are located in clear-sky regions as indicated by the model short-range forecast and the observations themselves. However, the largest decrease in the forecast error is provided by observations detected in cloudy regions. It is shown that the use of Special Sensor Microwave/Imager data helps to reduce model systematic errors in the central eastern equatorial Pacific, where the intertropical convergence zone is located, and the Arabian Sea, where the monsoon season occurs. Copyright © 2011 Royal Meteorological Society

1. Introduction

Over the last decade, data assimilation schemes have evolved towards very sophisticated systems, such as the four-dimensional variational system (4D-Var) (Rabier et al., 2000) that operates at the European Centre for Medium-Range Weather Forecasts (ECMWF). The scheme handles a large variety of both space and surface-based meteorological observations. It combines the observations with prior (or background) information on the atmospheric state and uses a comprehensive (linearized) forecast model to ensure that the observations are given a dynamically realistic, as well as statistically likely, response in the analysis. Effective performance monitoring of such a complex system, with an order of 108 degrees of freedom and more than 107 observations per 12 h assimilation cycle, has become an absolute necessity.

The assessment of the contribution of each observation to the analysis is among one of the most challenging diagnostics in data assimilation and numerical weather prediction. Methods have been derived to measure the observational influence in data assimilation schemes (Purser and Huang, 1993; Fisher, 2003; Cardinali et al., 2004; Chapnick et al., 2004). These techniques therefore show how the influence is assigned during the assimilation procedure, which partition is given to the observation and which is given to the background or pseudo-observation. They therefore provide an indication of the robustness of the fit between model and observations and allow some tuning of the weights assigned in the assimilation system. Measures of the observational influence are useful for understanding the data assimilation (DA) scheme itself. How large is the influence of the latest data on the analysis and how much influence is due to the background? How much would the analysis change if one single influential observation were removed? How much information is extracted from the available data?

To answer these questions, it was necessary to consider the diagnostic methods that have been developed for monitoring statistical multiple regression analyses; 4D-Var is, in fact, a special case of the generalized least square (GLS) problem (Talagrand, 1997) for weighted regression, thoroughly investigated in the statistical literature.

Recently, adjoint-based observation sensitivity techniques have been used (Baker and Daley, 2000; Cardinali and Buizza, 2004; Langland and Baker, 2004; Morneau etal., 2006; Zhu and Gelaro, 2008; Cardinali, 2009) to measure the observation contribution to the forecast, where the observation impact is evaluated with respect to a scalar function representing the short-range forecast error. In general, the adjoint methodology can be used to estimate the sensitivity measure with respect to any parameter of importance of the assimilation system. Very recently, Daescu (2008) and Daescu and Todling (2010) derived a sensitivity equation of an unconstrained variational DA system from the first-order necessary condition with respect to the main input parameters: observation, background and their error covariance matrices. The paper provides the theoretical framework for further diagnostic tool development not only to evaluate the observation impact on the forecast but also the impact of the other analysis parameters. Sensitivity to background covariance matrix can help in evaluating the correct specification of the background weight and their correlation. Limitations and weaknesses of the covariance matrices are well known, and several assumptions and simplifications are made to derive them. Desroziers and Ivanov (2001) and Chapnik et al. (2006) discussed the importance of diagnosing and tuning the error variances in a data assimilation scheme.

The adjoint-based observation sensitivity technique measures the impact of observations when the entire observation dataset is present in the assimilation system. It also measures the response of a single forecast metric to all perturbations of the observing system. It provides the impact of all observations assimilated at a single analysis time.

The adjoint-based technique is limited by the tangent linear assumption, valid up to 3 days. Furthermore, a simplified adjoint model is usually used to carry the forecast error information backwards, which further limits the validity of the linear assumption, and therefore restricts the use of the diagnostic to a typical forecast range of 24–48 h.

In this paper, a comprehensive assessment of the impact of all-sky microwave imager radiances in the assimilation and forecast system is obtained by applying the diagnostic tools introduced above. All-sky assimilation of microwave imager radiances became operational in March 2009 (Bauer et al., 2010; Geer et al., 2010). In the all-sky approach, radiance observations from the Special Sensor Microwave/Imager (SSM/I; Hollinger et al., 1990) and Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E; Kawanishi et al., 2003) are assimilated under all atmospheric conditions, whether clear, cloudy or rainy. At the frequencies used by microwave imagers, the atmosphere is semi-transparent except in heavy cloud and precipitation conditions. Observations are only assimilated over oceans, where the observations are sensitive to ocean surface properties (e.g. surface temperature and wind speed), atmospheric water vapour, cloud, hydrometeor profiles and precipitation.

Recently, a complete revision of observation error variance, quality control, thinning and resolution matching of the all-sky assimilation of microwave imagers was implemented in the operational ECMWF 4D-Var system (Bauer et al., 2010; Geer et al., 2010).

As defined, observation error variance and bias correction make use of information from both model forecast and observations. Because cloud and precipitation structures are often misplaced in the model forecast, a variational bias correction that is only based on model information would often misrepresent cases where model and observations disagree on the sky representation. Therefore, the observation error variances and the observation bias correction are, in the new approach, a function of the mean cloud amount as indicated by the model and the observations (Geer and Bauer, 2010). The spatial scale of the observations has also been examined. Instead of taking the nearest single all-sky observation to a grid point, an average or ‘superob’ of all observations falling into a grid box is calculated prior to the assimilation. Also, the new approach screens out observations where the model shows very cold and heavy snowfall biases. These biases are in fact too difficult to deal with using a predictor-based bias correction scheme.

The paper is organized as follows: sections 2 and 3 describe the mathematical framework of the adjoint tools used in the investigation. Section 4 presents the results, and conclusions are provided in section 5.

2. Analysis and forecast observational influence for DA scheme

DA systems for NWP provide estimates of the atmospheric state x by combining meteorological observations y with prior (or background) information xb. A simple Bayesian Normal model provides the solution as the posterior expectation for x, given y and xb. The same solution can be achieved from a classical frequentist approach, based on a statistical linear analysis scheme providing the best linear unbiased estimate (Talagrand, 1997) of x, given y and xb. The optimal GLS solution to the analysis problem (see Lorenc, 1986) can be written

equation image(1)

The vector xa is the ‘analysis’. The gain matrix K (n × p) takes into account the respective accuracies of the background vector xb and the observation vector y as defined by the n × n covariance matrix B and the p × p covariance matrix R, with

equation image(2)

Here, H is a p × n matrix interpolating the background fields to the observation locations, and transforming the model variables to observed quantities (e.g. radiative transfer calculations transforming the model's temperature, humidity and ozone into brightness temperatures as observed by several satellite instruments). In the 4D-Var context introduced below, H is defined to include also the propagation in time of the atmospheric state vector to the observation times using a forecast model.

Substituting (2) into (1) and projecting the analysis estimate onto the observation space, the estimate becomes

equation image(3)

It can be seen that the analysis state in observation space (Hxa) is defined as a sum of the background (in observation space, Hxb) and the observations y, weighted by the p × p square matrices IHK and HK, respectively.

In this case, for each unknown component of Hx, there are two data values: a real and a ‘pseudo’ observation. The additional term in (3) includes these pseudo-observations, representing prior knowledge provided by the observation-space background Hxb. The analysis sensitivity with respect to the observations is obtained (Cardinali et al., 2004):

equation image(4)

We focus here on (4). The (projected) background influence is complementary to the observation influence. For example, if the self-sensitivity with respect to the ith observation is Sii, the sensitivity with respect the background projected onto the same variable, location and time will be simply 1 − Sii.

In particular, the observation influence Sii = [0,1] where Sii = 0 means that the ith observation has had no influence at all in the analysis (only the background counted) and Sii = 1 indicates that an entire degree of freedom has been devoted to fit that data point (the background has had no influence). The tr(S) can be interpreted as a measure of the amount of information extracted from the observation or ‘degree of freedom for signal’ (DFS), while it follows that the complementary trace, tr(IS) = p − tr(S), is the DF for background. That is the weight given to prior information, to be compared to the observational weight tr(S). Hereafter, the observation influence Sii is denoted OI. In conclusion, DFS is a function of the observation and the background covariance matrices, the model itself as a time-spatial propagator and the number of observations.

3. Forecast sensitivity to the observations

Baker and Daley (2000) derived the forecast sensitivity equation with respect to the observations in the context of variational DA. Let us consider a scalar J-function of the forecast error. Then, the sensitivity of J with respect to the observations can be written using a simple derivative chain as

equation image(5)

∂J/∂xa is the sensitivity of forecast error to initial condition xa (Rabier et al., 1996; Gelaro et al., 1998), where the forecast error is expressed as dry energy norm. A few years ago, the use of moist norm instead of the dry one was investigated (Barkmeijer et al., 2001) and results have indicated that if a humidity term is considered in the final time norm the largest norm contribution with respect to the initial analysed fields was unrealistically provided by humidity rather than by vorticity, divergence or temperature fields. It was therefore necessary to apply an arbitrary tuning coefficient to diminish the effect. A full representation of the moist processes in the adjoint model is instead used that appropriately links the sensitivity of the forecast error with respect to the initial humidity with the sensitivity with respect to the other fields, e.g. temperature. A further investigation on the subject is planned at ECMWF (Janiskova, personal communication).

From (1) the sensitivity of the analysis system with respect to the observations and the background can be derived from

equation image(6)

By using (6) and (2) the forecast sensitivity to the observations becomes

equation image(7)

A second-order sensitivity gradient needs to be considered in (7) (Langland and Baker, 2004; Errico, 2007) because only superior orders other than first contain the information related to the forecast error. In fact, the first-order one only contains information on the suboptimality of the assimilation system (Cardinali, 2009).

The variation δJ of the forecast error expressed by J can be found by rearranging (5) and by using the adjoint property for the linear operator:

equation image(8)

where δxa = xaxb are the analysis increments and δy = yHxb is the innovation vector. The sensitivity gradient equation image is valid at the start time of the 4D-Var window (typically 0900 and 2100 UTC for the 12 h 4D-Var set-up used at ECMWF). As for K, its adjoint KT incorporates the temporal dimension, and the δy innovations are distributed over the 12 h window. The variation of the forecast error due to a specific measurement can be summed up over time and space in different subsets to compute the average contribution of different components of the observing system to the forecast error. For example, the contribution of all AMSU-A satellite instruments, s, and channels, i, over time t will be

equation image(9)

The forecast error contribution can be gathered over different subsets that can represent a specific observation type, a specific vertical or horizontal domain, or a particular meteorological variable. In summary, the forecast error contribution to each measurement assimilated depends on the sensitivity to the forecast error with respect to the measurement (large absolute forecast error determines large absolute sensitivity), the adjoint of the assimilation system (which is affected by the background and the observations statistics as the model itself) and the innovation vector. When the measurements are gathered together, e.g. by instrument type, also the number will affect the observation impact.

4. Results

Analysis and forecast experiments using the ECMWF 4D-Var system (Rabier etal., 2000; Janiskova et al., 2002; Lopez and Moreau, 2005) have been performed for June 2009 to assess, in particular, the impact of SSM/I and AMSR-E microwave imager observations, sensitive to humidity, cloud, precipitation and ocean surface (Geer and Bauer, 2010). The 24 h forecast error contribution (FEC) of all the observing system components is computed and shown in Figure 1. The largest contribution in decreasing the forecast error is provided by AMSU-A (∼16%), IASI and AIRS (12%), followed by AIREP (aircraft data 10%), TEMP (radiosonde 7%), GPS-RO (6%) and SCAT data (5%). All the other observations contribute up to 4%. SSM/I decreases the 24 h forecast error by 4% and the AMSR-E by 2%, SSM/I being the largest contribution among radiance data sensitive to humidity. Table I summarizes the window channels from the two microwave sensors used in this study. SSM/I is a conical scanning passive microwave imager on board the latest generation of DMSP satellites since June 1987. Observations are acquired at four frequencies (19.3, 22.2, 37.0 and 85.5 GHz), with a dual polarization (only horizontal at 22.2 GHz) and near-constant zenith angle of 53°. The instrument makes measurements at the mean altitude of 830 km with a swath width of 1400 km and horizontal resolution from 12.5 km (at 85.5 GHz) to 25 km (at 19.3 GHz). In addition to SSM/I, AMSR-E data have been assimilated. The AMSR-E instrument has been operating aboard NASA's Aqua satellite since 4 May 2002. It is a 12-channel, six-frequency, passive-microwave radiometer system with a near-constant zenith angle of about 55°. It measures horizontally and vertically polarized brightness temperatures at 6.9, 10.7, 18.7, 23.8, 36.5, and 89.0 GHz. At an altitude of 705 km, AMSR-E measures the upwelling scene brightness temperatures with a swath width of 1445 km, and the horizontal resolution varies from 16 to 27 km at 18.7 GHz and from 8 to 14 km at 36.5 GHz (Kawanishi et al., 2003).

Figure 1.

24 h forecast error contribution as a percentage for June 2009 and all observation types assimilated.

Table I. Window channels from microwave sensors.
ChannelFrequency (GHz)Polarization

The observation influence of microwave imager data can also be computed and compared with the forecast error reduction for all the channels assimilated. Figure 2 shows the DFS (tr(S), (a)) and FEC (b) for the five SSM/I channels used and for the different flags applied to the observations.

Figure 2.

(a) DFS and (b) FEC as percentage per SSM/I assimilated channels.

The flags are given during the assimilation process accordingly to the model background (12 h forecast) and observation cloudiness information: flag-Clear and flag-Cloudy means that model and observation are consistently indicating that the measurement is in a clear-sky or in cloudy-sky area, respectively. Flag-OBS-Clear and OBS-Cloudy indicate that observation and first guest provide different information on cloudiness: whereas the observation denotes a clear-sky measurement, the first guess at the same location registered clouds and vice versa (Geer and Bauer, 2010).

From Figure 2(a), the largest contribution in the analysis is provided by flag-Clear from a maximum of 18% (channel 1) to 5% (channel 6). The first two SSM/I channels have strong sensitivity to surface wind, while channels 3 and 6 are dominated by water vapour. All the other flags show a very similar DFS that varies from 5% (channel 1) to 1% (channel 6). Flag-Clear (both observation and background in clear sky) also presents the largest observation influence (Sii or mean influence per measurement) for all channels (not shown), while all the other flags exhibit a different observation influence ranking with respect to the DFS. In particular, flag-Cloudy, with the second largest DFS, has the least observation influence (0.15), indicating that the DFS was in this flag-case affected more by the number of observations (larger than for OBS-Clear or OBS-Cloudy flags) than by the single measurement influence. Measurements in cloudy areas are expected to be less influential since, for example, the observation error variance is assumed to increase proportionally with the cloud amount. On the 24 h forecast impact, the largest contribution to the decrease of forecast error is due to flag-Cloudy observations (Figure 2(b)) and in particular from a maximum of ∼12% (channel 3) to a minimum of 3% (channels 6 and 4). The smallest forecast contribution is provided by observations flagged OBS-Clear that also have the smallest DFS.

Similar impact in the analysis and the short-range forecast is noticed for AMSR-E radiances. In particular, for AMSR-E, the per channel impact distribution is different, likely due to the fact that the instrument also has a horizontally polarized 24 GHz channel with strong sensitivity to water vapour which enhances the contribution of both channels 7 and 8. Figure 3 show DFS (a) and FEC (b) for all AMSR-E channels assimilated. The largest DFS is provided by flag-Clear observations (∼17%, channel 8), while the largest forecast error decrease is due to flag-Cloudy observations (∼17%, channel 8). The smallest DFS and FEC contributions are from observations flagged OBS-Cloudy and OBS-Clear, respectively.

Figure 3.

(a) DFS and (b) FEC as percentage per AMSR-E assimilated channels.

In general, the assimilation of humidity-sensitive radiances from microwave instruments shows the largest analysis influence when observations together with the first-guess indicate that the measurement is taken in clear sky areas, but the largest contribution to the 24 h forecast error comes instead from observations taken (according to the measurement and first-guess indication) in cloudy areas. Several reasons can explain this: cloudy areas are believed to be the regions where the forecast error is growing faster; consequently, observations can contribute more towards reducing the error than in other regions where the forecast error is much smaller. Both DFS and FEC depend on the transpose Kalman gain matrix KT; FEC also depends on the forecast error and in particular FEC is modulated by the percentage of forecast error that projects on KT. Errors in the verification analysis (the analysis used to compute the forecast error) would underestimate or overestimate the forecast error and its projection on KT and the total effect will be manifest on a decrease or increase (respectively) of impact with respect to the DFS. In numerical weather prediction, since the ‘truth’ is unknown, the analysis is used, being the best representation of the truth, to compute the forecast error. In the near future satellite data, providing a homogeneous and dense data coverage, could also be used to provide a independent forecast error estimation.

Observation influence and forecast impact with respect to a particular data type can be geographically mapped. Figure 4(a) shows the average observation influence for SSM/I radiances and for all the cloudiness flags considered. The largest contribution in the analysis is observed in the tropical and subtropical (for the Northern Hemisphere) band and follows the distribution of water vapour and precipitation: over the Pacific Ocean along the mean position of the intertropical convergence zone (ITCZ) (OI = 0.25), the Gulf of Mexico (OI = 0.38) and the Indian summer monsoon region (OI = 0.35–0.4). No significant differences on observation influence patterns are observed among the different data-flags, i.e. in clear skies the larger observation sensitivities follow the water vapour patterns. FEC is illustrated in Figure 4(b), where negative (blue shading contour) and positive (red shading colour) values mean decrease and increase of forecast error, respectively. A quite consistent reduction of 24 h forecast error is observed almost everywhere. Some forecast degradation (positive values) areas can be noticed over the western Pacific, the Indian Ocean and on the Pacific Ocean close to the South American continent. Interestingly, different flag-types correspond to different FEC patterns (not shown). In particular, the all-sky flags pattern (Figure 4(b)) is very similar to the OBS-Cloudy flag pattern for the tropical band.

Figure 4.

(a) SSM/I All-sky radiances OI (non-dimensional units) and (b) FEC (joules) averaged over June 2009.

Two areas of forecast impact will be analyzed in more detail: the central eastern equatorial Pacific, where the ITCZ is located, and the Arabian Sea, where the monsoon season occurs. From Figure 4(b) SSM/I largely contributes to decrease the forecast error along the mean position of ITCZ. To assess the specific impact of SSM/I in the areas of interest as highlighted by Figure 4(b), an observation system experiment (OSE) has been performed with and without the assimilation of these data. Figure 5 shows the zonal average cross-section between 150° to 110° west of the mean analysis differences with and without SSMI/I for June 2009. SSM/I reduces the amount of humidity at low level (∼850 hPa) by a maximum of 4%, while near the surface the relative humidity content increases on average by 2% (Figure 5(a)) from 0° to 20°N. The indirect impact on the circulation can be seen in Figure 5(b), where the vertical velocity cross-section mean differences (10−2 Pa/s) indicate an intensification of the rising motion, associated with the mean position of the ITCZ. In particular, the main convective cell around 8°N depicts an increased upward motion of 0.04 Pa/s that results in an intensification of the circulation convergence in the lower atmosphere (not shown). The total SSM/I effect is therefore to reinforce the dominant branch of the Hadley circulation in the Northern Hemisphere, which is known to be persistently weak in the ECMWF forecast model (Rodwell and Jung, 2008).

Figure 5.

Zonal average cross-section (150–110°W) from 1000 to 100 hPa of the mean analysis differences with and without assimilation of SSM/I for June 2009: (a) relative humidity (%); (b) vertical velocity (10−2 Pa/s).

In the Arabian Sea, June is dominated by the initiation of the Indian summer monsoon circulation. From Figure 4(b), a general decrease of the 24 h forecast error is noticed, with a larger impact close to the Arabic peninsula and the Indian continent. The OSE in that area shows an average reduction in relative humidity up to 6% in the troposphere, below 500 hPa, between the Equator and 18°N (Figure 6(a)) when SSM/I microwave radiances are assimilated.

Figure 6.

Zonal average cross-section (50–71°E) from 1000 to 100 hPa of the mean analysis differences with and without assimilation of SSM/I for June 2009: (a) relative humidity (%); (b) zonal wind (10−2 m s−1).

The indirect effect on the wind field can be summarized from Figure 6(b) (meridional wind not shown). The mean differences indicate a stronger near-surface southwesterly wind when SSM/I is included. In the low troposphere (∼850 hPa) the analysed winds tend to increase close to the Equator, while the westerly component of the wind is reduced in the region above 12°N. The overall effect is then to reduce, in the ECMWF forecast model, the monsoon circulation where it is too strong, e.g. across the Arabian Sea (Rodwell and Jung, 2008).

5. Conclusions

Over the last few years, the potential of using derived adjoint-based diagnostic tools has been increasingly exploited.

The influence matrix is a well-known concept in multivariate linear regression, where it is used to identify influential data and to predict the impact on the initial condition estimates of removing individual data from the regression. The self-sensitivity provides a quantitative measure of the observation influence in the analysis. In the context of 4D-Var there are many components that together determine the influence given to any one particular observation. First there is the specified observation error covariance R, which is usually well known and obtained simply from tabulated values. Second, there is the background error covariance B, and third, the dynamics and the physics of the forecast model which propagate the covariance in time, and modify it according to local error growth in the prediction. The influence is further modulated by data density.

Forecast sensitivity to observations can be used to diagnose the impact on the short-range forecast, namely 24–48 h, given the use of a simplified adjoint of the DA system and the implied linearity assumption. Forecast error contribution maps allow the geographical identification of beneficial or detrimental observation impact, and a clear understanding of the causes can be drawn with the help of the OSE, in which the data of interest are denied. In general, OSEs are also used to investigate forecast data impact on a longer-range forecast, typically 5 days.

The global impact of observations is found to be positive and the forecast errors decrease for all data types. The largest contribution is provided by microwave sounder radiances (AMSU-A), followed by infrared sounder radiances (IASI and AIRS) from the instruments that mainly provide information on temperature. For microwave satellite humidity information, SSM/I (microwave imager), AMSU-B (microwave sounder), AMSR-E (microwave imager) and MHS (microwave sounder) instruments are in this order contributing to forecast error decrease. In particular, the recent changes applied to the assimilation of all-sky microwave imager radiances increased by a factor of three their mean influence in the analysis (see Geer and Bauer, 2010; Geer et al., 2010) and consequently their impact on the short-range forecast. Since the observation influence decreases with the increase of mean amount of clouds, as indicated by model and observations, the most penalized information in the analysis is in a very cloudy region, while double DFS is measured on clear-sky ones. Interestingly, in cloudy areas, microwave imager information is able to decrease the forecast error twice as much as in clear-sky conditions. It is believed that the largest impact in these areas is mainly due to the larger error present in the forecast in high baroclinic cloudy regions.

OSEs, used to understand the direct (on the observable humidity field) and indirect (on wind field) effect of the assimilation of SSM/I data in specific beneficial areas as indicated by the forecast error contribution maps, highlight that in the Arabian Sea during the monsoon circulation an average reduction in relative humidity up to 6% on the lower troposphere took place, with a consequent reduction of the too strong Monsoon circulation. Over the central Pacific ITCZ, SSM/I has reduced the humidity amount in the upper boundary layer (4%) and increased near surface (2%), causing an intensification of the Hadley circulation that is well known to be too weak in the ECMWF forecast.


The authors thank Mohamed Dahoui for graphical support and Alan Geer, who kindly performed the observation system experiment. Many thanks also to Peter Bauer for the useful discussion on the observation characteristics, and to Jean-Noel Thépaut and two anonymous reviewers for improving the manuscript.