Error covariance sensitivity and impact estimation with adjoint 4D-Var: theoretical aspects and first applications to NAVDAS-AR

Authors

  • Dacian N. Daescu,

    Corresponding author
    1. Portland State University, Portland, OR, USA
    • Portland State University, PO Box 751, Portland, OR 97207, USA.
    Search for more papers by this author
  • Rolf H. Langland

    1. Naval Research Laboratory, Marine Meteorology Division, Monterey, CA, USA
    Search for more papers by this author
    • The contribution of this author to this article was prepared as part of his official duties as a United States Federal Government employee.


Abstract

This article presents the adjoint-data assimilation system (adjoint-DAS) approach to evaluate the forecast sensitivity with respect to the specification of the observation-error covariance (R-sensitivity) and background-error covariance (B-sensitivity) in a four-dimensional variational (4D-Var) DAS with a single outer-loop iteration. Computationally efficient estimates to the forecast impact of adjustments in the error covariance models are obtained by exploiting the mathematical properties of the R- and B-sensitivity matrices and their relationship with the observation sensitivity vector. An additional contribution of this work is that it establishes a synergistic link between various methodologies to analyze the DAS performance: observation sensitivity and impact assessment, error covariance sensitivity, and a posteriori diagnosis. The practical ability to obtain sensitivity information with respect to R- and B-parameters is presented with the adjoint versions of the Naval Research Laboratory Atmospheric Variational Data Assimilation System–Accelerated Representer (NAVDAS-AR) and the Navy Operational Global Atmospheric Prediction System (NOGAPS). The adjoint approach is used to provide guidance on the forecast impact of weighting the radiance data in the DAS according to observation-error variance estimates derived from an a posteriori diagnosis. The results indicate that information extracted from both error covariance diagnosis and sensitivity analysis is necessary to design parameter tuning procedures that are effective in reducing the forecast errors. Copyright © 2012 Royal Meteorological Society

1. Introduction

Atmospheric data assimilation systems (DASs) rely on the state estimation theory to ingest the information content of observations into numerical weather prediction (NWP) models (Jazwinski, 1970; Lahoz et al., 2010). The observation performance in reducing the analysis and forecast errors is closely determined by the representation in the DAS of the statistical properties of the errors in the prior state estimate, model, and observations. Consistency diagnostics for the covariances of the observation and background errors may be obtained a posteriori from the statistical analysis of the DAS products (Talagrand, 1999; Desroziers et al., 2005). Diagnosis studies indicate that both spatial and inter-channel error correlations are present in the satellite radiances provided by the atmospheric sounders (Garand et al., 2007; Bormann and Bauer, 2010; Bormann et al., 2010, 2011; Gorin and Tsyrulnikov, 2011) and additional error correlations are introduced when order reduction techniques are used to extract the information content of high-resolution datasets (Collard et al., 2010).

A common approach taken at NWP centres is to assign a diagonal weight matrix to the observational component, and error variance inflation is performed to compensate for unrepresented error correlations. Quantification of the loss of information as a result of suboptimal weighting, and implementation of efficient procedures to adjust the error covariance parameters to a configuration that improves the forecasts' skill, are areas of active research. Synergistic efforts include the development of error covariance models for NWP applications (Derber and Bouttier, 1999; Gaspari and Cohn, 1999; Lorenc, 2003; Fisher, 2003; Bannister, 2008a, 2008b; Frehlich, 2011; Bishop et al., 2011; Raynaud et al., 2011) and of computationally feasible techniques for diagnosis, estimation, and tuning of the error covariance parameters (Wahba et al., 1995; Dee, 1995; Dee and Da Silva, 1999; Andersson et al., 2000; Desroziers and Ivanov, 2001; Cardinali et al., 2004; Buehner et al., 2005; Chapnik et al., 2006; Zupanski and Zupanski, 2006; Trémolet, 2007; Anderson, 2007; Liu and Kalnay, 2008; Desroziers et al., 2009; Li et al., 2009).

Observing system experiments (OSEs) are the traditional tool to assess the observation value (Atlas, 1997; Kelly et al., 2007) and, to date, studies on the forecast impact as a result of variations in the specification of the observation- and background-error covariance parameters have been only performed through additional data assimilation experiments (Zhang and Anderson, 2003; Joiner et al., 2007). OSEs allow the investigation of only a few parameters in the DAS since a new experiment is required for each parameter input. The assimilation of radiances from hyperspectral instruments such as the Atmospheric Infrared Sounder (AIRS) and the Infrared Atmospheric Sounding Interferometer (IASI) has increased the number of ‘tunable’ parameters to a dimension where an analysis by trial-and-error is not feasible. Computationally efficient techniques are necessary to identify those error covariance parameters of potentially large forecast impact and to obtain insight on the performance of a new error covariance model prior to its actual implementation in the DAS i.e. without performing an additional assimilation experiment. The adjoint approach to parameter sensitivity and impact estimation provides a basis to advance research in this area.

Baker and Daley (2000) have shown that an all-at-once evaluation of the forecast sensitivity to observations may be performed by developing the adjoint of the data assimilation system (adjoint-DAS). The analysis of the information content of observations and the observation impact assessment through observation sensitivity and adjoint-DAS techniques are routine activities at NWP centres to monitor the observing system performance on reducing the short-range forecast errors (Langland and Baker, 2004; Trémolet, 2008; Baker and Langland, 2009; Cardinali, 2009; Daescu and Todling, 2009; Gelaro and Zhu, 2009; Gelaro et al., 2010; Cardinali and Prates, 2011; Lupu et al., 2011). The adjoint-DAS applications may be extended to incorporate the sensitivity analysis with respect to error covariance parameters and the estimation of the forecast impact from adjusting the error covariance models. Daescu (2008) derived the equations of the forecast sensitivity with respect to the observation-error covariance model (R-sensitivity) and to the background-error covariance model (B-sensitivity) in a nonlinear four-dimensional variational (4D-Var) DAS. The practical ability to estimate the forecast sensitivity to R- and B-weight parameters in a 3D-Var DAS was shown by Daescu and Todling (2010).

The current work presents novel theoretical aspects of the adjoint-DAS error covariance sensitivity analysis and first applications in an operational 4D-Var data assimilation and forecast system, the Naval Research Laboratory Atmospheric Variational Data Assimilation System–Accelerated Representer (NAVDAS-AR; Daley and Barker, 2001; Xu et al., 2005; Rosmond and Xu, 2006) and the Navy Operational Global Atmospheric Prediction System (NOGAPS; Hogan and Rosmond, 1991).

The article is organized as follows. A review of the analysis equation in a strong-constraint 4D-Var implementing a single outer-loop iteration is put forward in section 2 to elucidate the notational convenience and the four-dimensional structure of the operators involved in the error covariance sensitivity estimation. Section 3 provides the derivation of the forecast R- and B-sensitivity equations consistent with the analysis scheme. Special care is taken to account for the dependence of the linearized model and the linearized observation operator on the background forecast, and it is shown that both R- and B-sensitivities may be evaluated without the need of a second-order adjoint model. Section 4 includes additional sensitivity properties and applications of the adjoint-DAS approach as a guidance tool to error covariance parameter tuning and impact assessment. A computationally efficient procedure to evaluate the first-order forecast impact of a low-rank B-covariance update is presented. In section 5, numerical results obtained with NAVDAS-AR/NOGAPS are used to emphasize the complementary information derived from various techniques to analyze the DAS performance: observation impact, sensitivity to error covariance weighting, and error covariance diagnosis. First applications are also presented of the adjoint-DAS observation-error covariance sensitivity to provide guidance on the forecast impact of error variance estimates derived from an a posteriori diagnosis. A summary and further research perspectives are in section 6.

2. The analysis equation

The development of the NRL 4D-Var system NAVDAS-AR including the nonlinear formulation, implementation of outer-loop iterations, and the ability to account for model errors is discussed in the work of Xu et al. (2005) and Rosmond and Xu (2006). The analysis framework adopted in this study is of a strong-constraint 4D-Var DAS with a single outer-loop iteration. The nonlinear cost functional is defined as

equation image(1)

where equation image is a prior (background) estimate of the true state equation image at the initial time t0 of the assimilation interval [t0,tN], equation image is the model state vector at time tk, equation image, equation image is the observation vector at time tk, and equation image denotes the observational operator that maps the state xk into observations yk. Statistical information on the background error equation image and the observational errors equation image is used to specify the positive definite matrices equation image and equation image that are representations in the DAS of the background- and observational-error covariances Bt and equation image, respectively.

The model equations

equation image(2)

where ℳk−1,k denotes the nonlinear forecast model from tk−1 to tk, are imposed as a strong constraint (perfect model assumption) such that by expressing

equation image(3)

the only free variable in the minimization of the cost (1) is the model state x0 at the initial time t0.

The first outer-loop iteration of the minimization process involves two linear approximations. The first linearization is of the observation operator

equation image(4)

where equation image denotes the background forecast at time tk and

equation image(5)

is the Jacobian matrix of the observational operator hk evaluated at equation image. The second linearization is of the forecast model equations (3)

equation image(6)

where equation image denotes the tangent linear model from t0 to tk evaluated along the background trajectory

equation image(7)

After replacing (4) and (6) into (1), the analysis equation image at the initial time t0 is obtained by minimization of the quadratic cost functional

equation image(8)

where

equation image(9)

is the time-distributed innovation vector of dimension p = p0 + p1 + ··· + pN, the operator H incorporates both the linearized observation operator and the tangent linear model and is defined as

equation image(10)

and

equation image(11)

is the block diagonal observation-error covariance matrix. The analysis state at the initial time t0 minimizes the cost (8) and it is expressed as

equation image(12)

where the gain matrix (DAS operator) K is defined as

equation image(13)
equation image(14)

The observation-space evaluation of (12) is a two-stage process consisting of solving the linear system

equation image(15)

for the vector equation image, followed by a post-multiplication operation

equation image(16)

In NAVDAS-AR, the computational steps (15) and (16) are performed using a matrix-free implementation (Xu et al., 2005; Rosmond and Xu, 2006). From (6) and (7), it is noticed that a single integration of the tangent linear model (forward sweep) from t0 to tN is required to obtain the analysis at all intermediate time instants tk of the assimilation interval

equation image(17)

Combining (16) and (17), the analysis state at each tk,k = 0 : N, is formally expressed as

equation image(18)
equation image(19)

where

equation image(20)

denotes the gain matrix at time tk. At the initial time t0, equation image is the n × n identity matrix.

3. Forecast sensitivity to the R- and B-specification

The evaluation of the forecast sensitivity with respect to the error covariance models R and B is presented for a scalar aspect of the forecast initialized from the analysis equation image at time tk of the 4D-Var assimilation interval [t0, tN]. This general framework is considered to accommodate a variety of practical situations, e.g. in the observation impact studies performed with NAVDAS-AR the analysis equation image is valid at the middle of the 6 h assimilation interval, [t0, tN] = [tk − 3h, tk + 3h]. The forecast score is typically defined as a short-range forecast-error measure

equation image(21)

where equation image is the model forecast at verification time tf initiated from equation image, equation image is the verifying analysis at tf and serves as a proxy to the true state equation image, and E is a diagonal matrix of weights that gives (21) units of energy per unit mass.

The first-order variation in the forecast aspect equation image induced by the analysis variation equation image is defined as

equation image(22)

where 〈·,·〉 denotes the Euclidean inner product, 〈u,v〉 = vTu. For the functional (21), the forecast sensitivity to analysis is evaluated using a backward adjoint model integration from tf to tk along the analysis trajectory

equation image(23)

The forecast sensitivities to R and B are expressed, respectively, as the matrices of first-order partial derivatives

equation image(24)
equation image(25)

The analysis equations (12) to (20) provide the basis to evaluate the forecast R- and B-sensitivities and are used to express the first-order variation equation image in (22) in terms of variations δR and δB, respectively. A key role in the estimation process is played by the observation sensitivity vector (Baker and Daley, 2000) which is expressed from (19) and (22) as

equation image(26)

where equation image denotes the adjoint-DAS operator at tk. The evaluation of the observation sensitivity is integrated in the routine activities at NWP centres to monitor the observing system performance. In NAVDAS-AR, an all-at-once assessment of the observation impact on the forecast-error reduction is obtained with the second-order observation-space approximation measure (Langland and Baker, 2004)

equation image(27)

where g is the average of two forecast gradients evaluated along the analysis and background trajectories

equation image(28)

It is noticed that the observation-space inner product (27) incorporates all observations in the DAS. The forecast impact associated with any subset of observations yi is evaluated by taking the inner product between the innovation vector component [yh(xb)]i and the corresponding component equation image of the vector equation image.

3.1. Forecast R-sensitivity

From (15) and (18), the first-order variation equation image associated with the matrix perturbation equation image in the covariance model R is expressed as

equation image(29)
equation image(30)

The first-order variation δz is expressed from (29) as

equation image(31)

and the explicit relationship between equation image and δR is obtained by inserting (31) in (30) and using (20):

equation image(32)

By replacing (32) in (22), the first-order variation δe is expressed in terms of δR as:

equation image(33)
equation image(34)
equation image(35)

Equation (35) shows that the forecast R-sensitivity is the rank-one matrix

equation image(36)

with entries

equation image(37)

It is noticed that the observation sensitivity vector (26) and the vector z provide all the necessary information to perform the forecast R-sensitivity analysis.

If the covariance model R is specified as a diagonal matrix, equation image where σo,i denotes the observation-error standard deviation associated with the observation yi, then

equation image(38)

is the forecast sensitivity to the specification of the observation-error variance.

3.2. Forecast B-sensitivity

From (15) and (18), the first-order variation equation image associated with the matrix perturbation equation image in the covariance model B is expressed as

equation image(39)
equation image(40)

The first-order variation δz is expressed from (39) as

equation image(41)

such that by applying the post-multiplication operator equation image to (41) and, using (20),

equation image(42)

The explicit relationship between equation image and δB is obtained by inserting (42) in (40):

equation image(43)

After replacing (43) in (22), the first-order variation δe induced by δB is expressed as

equation image(44)

For notational convenience, let equation image denote the vector

equation image(45)
equation image(46)

With the notation (45), the first-order variation in the forecast (44) is expressed as

equation image(47)

such that the forecast B-sensitivity is the rank-one matrix

equation image(48)

with entries

equation image(49)

By comparison with the R-sensitivity equation (36), it is noticed that all information necessary to perform the forecast B-sensitivity analysis is contained in the vectors w, defined in (45)–(46), and HTz. Therefore, it is emphasized that, if the analysis state is derived from a single outer-loop iteration of a 4D-Var DAS, then the evaluation of the forecast R- and B-sensitivity may be performed without the need of higher-order derivatives (Daescu, 2008; Trémolet, 2008).

The sensitivity equations presented in this section were derived from the observation-space formulation of the analysis (15)–(16) and are expressed in terms of the residual vector z defined in (15). Mathematically equivalent equations which may be used in both observation-space and analysis-space DASs are obtained by expressing z from (15) and (16) as

equation image(50)

In practice, caution must be exercised in the interpretation of the sensitivity estimates since only an approximate solution to the linear system (15) is obtained using an iterative procedure such as the conjugate gradient method. The development of an adjoint-DAS operator fully consistent with the iterations performed in the analysis scheme is addressed in the study of Zhu and Gelaro (2008).

Remark 1. If both the forecast model ℳ and the observation operator h are linear, then the analysis state equation image at tk is expressed from (19) as

equation image(51)
equation image(52)

where M0,k, H and Kk are state-independent operators (independent of the equation image specification). In this linear context, the vector w defined in (45) may be identified with the forecast sensitivity to the background-state specification at t0, as introduced in 3D-Var (tk = t0) by Baker and Daley (2000):

equation image(53)

To properly account for nonlinearities in the model ℳ and in the observation operator h, an evaluation of the forecast equation image-sensitivity that is consistent with the analysis equation (19) requires second-order derivative information i.e. a second-order adjoint model (Le Dimet et al., 2002; Daescu, 2008).

4. Additional sensitivity properties and applications

Novel applications of the adjoint-DAS software tools may be investigated by properly exploiting the mathematical properties of the forecast R- and B-sensitivity equations, in particular their rank-one matrix structure and their relationship with the observation sensitivity vector.

4.1. Forecast sensitivity to error covariance weighting

Proper weighting of the background information and of the information provided by various observational data components yi,i = 1 : I, may be investigated through a parametric specification

equation image(54)

which is a common approach used to perform error covariance tuning (Desroziers et al., 2009). In the formulation (54), sb > 0 and equation image i = 1 : I, are non-dimensional scalar parameters and {yi, i = 1 : I} denotes a partition of the observational data y consisting of data components equation image, equation image, and whose observational errors are assumed to be uncorrelated. It is noticed that (11) corresponds to a partition of the observations by the observing time, whereas for the purpose of covariance tuning, the data partitioning may be performed in a different fashion, e.g. by observation type. The covariance specification (R,B) in the reference DAS corresponds to all weight parameters in (54) set to 1, i.e. sb = 1 and equation image. The forecast sensitivity to each observation-error covariance weight factor is expressed from (35) and (54) as

equation image(55)

An intrinsic property of the 4D-Var minimization problem is that the multiplication of all error covariances in the DAS by the same positive constant has no impact on the analysis. Consequently, the forecast sensitivities to the covariance weight coefficients satisfy the relationship (Daescu and Todling, 2010)

equation image(56)

From (55) and (56), the forecast sensitivity to the background-error covariance weight factor sb is expressed as

equation image(57)

The inner product (57) incorporates all entries of the observation sensitivity and Rz vectors. Therefore, the forecast sb-sensitivity value is independent on the observation grouping used to define the y-partition and gives an indication on the proper weighting between the background information and the information provided by the observing system as a whole.

The intermediate stage z of the observation-space analysis may be eliminated from the sensitivity equations (55) and (57) by expressing the product Rz as

equation image(58)

In this formulation, (55) and (57) are expressed in terms of the observation sensitivity and the residual (58) that is produced in both observation-space and analysis-space DASs. Since the 4D-Var analysis (16) relies on the linearization of the model (6) and of the observation operator (4) along the background trajectory, the linear approximation

equation image(59)

may be used to reconcile (55), (57), and (58) with the forecast equation image- and sb-sensitivity equations derived by Daescu and Todling (2010) in terms of the observation sensitivity and the residual yh(xa). For completeness, a direct derivation of the sb-sensitivity (57) from the first-order variation (47) is provided in the appendix.

The sb- and equation image-sensitivities identify those weight parameters that are of potentially large forecast impact and the derivative information describes the local behaviour of the forecast error as a function of the weight assigned to each error covariance component in the DAS. The sensitivity guidance to forecast-error reduction is that negative derivatives point towards covariance inflation (s > 1), whereas positive derivatives point towards covariance deflation (s < 1). It is noticed that the sensitivity analysis per se does not provide the optimal parameter values such that this information may be only used to gain insight for error covariance tuning procedures.

4.2. Forecast sensitivity to the R-correlation model

If the observation-error correlations are modelled in the DAS, then R = ΣoCoΣo, where Σo = diag(σo,i) denotes the diagonal matrix of observation-error standard deviations and equation image is the observation-error correlation model, a symmetric and positive semi-definite matrix with all diagonal entries of 1. By replacing

equation image(60)

in (35), the first-order forecast variation δe induced by a δCo-perturbation in the observation-error correlation model is expressed as

equation image(61)
equation image(62)

From (62), the forecast Co-sensitivity is the rank-one matrix

equation image(63)

with entries

equation image(64)

4.3. Forecast sensitivity to the B-correlation model

The specification of the background-error correlations is a key ingredient of the B-model, B = ΣbCbΣb, where Σb = diag(σb,i) denotes the diagonal matrix of background-error standard deviations and equation image is the background-error correlation model. By replacing

equation image(65)

in (47), the first-order forecast variation δe induced by a perturbation δCb in the background-error correlation model is expressed as

equation image(66)
equation image(67)

From (67), the forecast Cb-sensitivity is the rank-one matrix

equation image(68)

with entries

equation image(69)

Remark 2. The covariance models R and B as well as the correlation models Co and Cb are symmetric matrices and the δ-variations associated with each of these matrices must satisfy the symmetry constraint. Therefore, for all practical purposes, the sensitivity matrices may be identified with their symmetric part which for a square matrix X is defined as

equation image(70)

4.4. Error covariance impact estimation

The R- and B-derivative information obtained through the adjoint-DAS approach facilitates a first-order assessment of various equation image and equation image models and specifications of covariance parameters at a reduced computational effort. Each of the models equation image and equation image may incorporate a new specification of the error variances, a new error correlation model, or both. Of relevance to practical applications is the ability to provide insight into the forecast performance of a model (equation image, equation image), as compared to the reference model (R,B), without the increased computational cost associated with an additional assimilation experiment. As explained below, the adjoint-DAS evaluation of the first-order forecast impact of each of the equation image and equation image models is very efficient and relies mainly on the ability to provide the observation sensitivity vector.

4.4.1. equation image-impact estimation

A first-order assessment of the forecast performance of a covariance model equation image, as compared to the model R in the DAS, may be obtained by setting δR = equation imageR in (35) and requires only the additional ability to provide the matrix/vector product [δR]z:

equation image(71)

In particular, if the observation-error covariance models R and equation image are specified as diagonal matrices, equation image and equation image, then (71) is expressed as

equation image(72)

The right-hand side of (72) provides an all-at-once first-order assessment to the forecast impact of each individual variation equation image. This approach is implemented in section 5 to investigate the potential forecast benefit of the observation-error variance specification equation image derived from an a posteriori diagnosis.

4.4.2. equation image-impact estimation

An ensemble-based flow-dependent background-error covariance specification may improve the performance of variational DASs (Hamill and Snyder, 2000; Buehner, 2005; Raynaud et al., 2011: Brousseau et al., 2012). A new background-error covariance model equation image may be formulated as

equation image(73)

In (73), Bd is a low-rank matrix that aims to capture the dynamic structure of the ‘errors of the day’,

equation image(74)

where the vectors equation image may be obtained from an ensemble of forecasts, and α is a coefficient used to weight the static and the dynamic components of the covariance model equation image, 0 ≤ α ≤ 1.

The operator format of the B-sensitivity (47) provides an efficient approach for assessing the forecast impact of each individual ensemble member of Bd. A first-order estimate to the forecast performance of the background-error covariance model equation image, as compared to the model B in the DAS, may be obtained by replacing

equation image(75)

in (47):

equation image(76)

The evaluation of each term in the right-hand side of (76) is detailed below. The first term is the first-order impact of a variation δsb = −α in the B-weight coefficient from sb = 1 to sb = 1 − α and may be evaluated using (57)/(86) as

equation image(77)

Each term equation image in (76) is a scalar quantity that may be evaluated as an observation-space Euclidean inner product

equation image(78)

By inserting (77) and (78) in (76), the first-order approximation to the forecast impact is expressed as

equation image(79)

Using (46), each equation image-inner product of (79) is evaluated as

equation image(80)

Equations (78), (79), and (80) reveal that, having available the observation sensitivity and the analysis sensitivity vectors, the evaluation of the first-order forecast impact of the ensemble member ui of the covariance model (73)–(74) requires only a tangent linear model integration (10) to obtain the vectors Hui and equation image and followed by a few inner-product operations. The assessment procedure is thus computationally efficient and may be performed simultaneously for a large number of vectors ui using parallel computing resources. This approach may be used in a hybrid variational-ensemble DAS to provide insight on the selection of the ensemble members for covariance updates, the ensemble size, and α-weight specification. From (79), it is noticed that the forecast sensitivity to the weight coefficient α is expressed as

equation image(81)

5. Results with the adjoint NAVDAS-AR/ NOGAPS

The adjoint-DAS software developed at NRL for observation sensitivity and observation impact assessment facilitates the estimation of the forecast sensitivity to R and B parameters. Results obtained with NAVDAS-AR/NOGAPS and their adjoint versions illustrate the complementary information provided by various tools for analyzing the DAS performance. Several operational upgrades were implemented in NAVDAS-AR during September 2010, including the assimilation of Global Positioning Satellite (GPS) bending-angle observations and of an increased amount of radiances by the addition to the assimilation system of the stratospheric channels 11–14 of the Advanced Microwave Sounding Unit (AMSU)-A, channels 22–44 of the Special Sensor Microwave Imager Sounder (SSMIS), and channels 122, 128, 135, 141, 148, 154, 161, 173, 185, 187 of IASI. Observation impact (OBSI) and forecast-error sensitivity to covariance weight parameters were evaluated for the 0000 UTC NAVDAS-AR analyses and the 24 h NOGAPS forecasts produced during the period 15 July–15 August 2010 (hereafter the summer period) and during the period 29 September–26 October 2010 (hereafter the autumn period) when the operational upgrades were already completed. NAVDAS-AR assimilates measurements of atmospheric temperature (T), zonal (u) and meridional (v) wind speed, relative humidity (rh), surface pressure p, surface-winds (Sfcwind), radiance, total precipitable water (tpw), and bending angle (autumn period only). A summary of the observation types and the associated observed parameters is provided in Table 1.

Table 1. Summary of the NAVDAS-AR observation types and the associated observed parameters.
NAVDAS-AR observation categoryObserved parameter
Land and Ship surfaceT,u,v,rh,p
Aircraft–AIREPS, AMDAR, MIDCRST,u,v,rh
Upper-level satellite windsu,v
Satellite Surface Winds–Scatterometer, Windsat, SSMIu,v,Sfcwind
Satellite Microwave Sounders AMSU-A, SSMISRadiance
Satellite Infrared Sounders AIRS, IASIRadiance
Satellite Moisture–SSMI/S, WindSattpw
Radiosondes and dropsondesT,u,v,rh,p
Synthetic–TC Bogus and Australian surface pressureT,u,v,p
Satellite Radio Occultation–GPS-ROBending angle

The routine procedure to OBSI assessment in NAVDAS-AR relies on (27)–(28) and it is implemented by applying the adjoint-DAS operator to the vector (28) which is defined in terms of both the analysis and background trajectories. An additional adjoint-DAS integration involving only the analysis trajectory is necessary to evaluate the observation sensitivity according to (23) and (26). This additional computation was performed for the purpose of sensitivity analysis; however, in a few of the analysis/forecast cycles, the observation sensitivity files were not properly formatted. Results of OBSI and sensitivity were obtained from the analysis of 30 datasets over the summer period (data for 28 July and 1 August were not incorporated in the study) and of 27 datasets over the autumn period (data for October 15 were not incorporated in the study). For consistency, the numerical results for each time period are presented as average values per assimilation/forecast cycle or as average values per observation.

The 0000 UTC analyses are obtained from the 4D-Var assimilation of observations in the 6 h time window from 2100 UTC to 0300 UTC and the forecast error measure considered is the 24 h global forecast error measured in a moist total energy norm (J kg−1). For each observed parameter, the number of observations assimilated in NAVDAS-AR to produce the 0000 UTC analyses is displayed in Figure 1 together with the associated OBSI on the forecast-error reduction, as derived from the second-order approximation (27)–(28). The fall period incorporates a significantly larger amount of radiances and in addition, it incorporates the assimilation of the GPS Radio Occultation data. Radiances have a substantial contribution to the forecast-error reduction whereas, when taking into account the data volume, it is noticed that measurements associated with other atmospheric parameters such as u- and v-wind speed, temperature, humidity, and pressure have a larger impact per observation. A detailed study on OBSI assessment at NWP centres is provided by Gelaro et al. (2010). In the following, we focus on the forecast R- and B-sensitivity guidance to covariance weight adjustments which may further reduce the forecast errors. By analogy with the OBSI, the sensitivity information is obtained all-at-once and allows the analysis of each observation type, instrument, and data location in the time–space domain.

Figure 1.

(a) The number of observations assimilated in NAVDAS-AR during the summer and autumn periods to produce the 0000 UTC analyses. (b) The observation impact (J kg−1) associated with each observed parameter on the 24 h forecast error reduction. Both graphs display the average values per assimilation/forecast cycle. The abbreviations for the observed parameters along the vertical axis denote: GPS–bending angle from GPS-RO; tpwater–total precipitable water; Radiance–satellite radiances; SfcPress–surface pressure; Sfcwind–satellite surface wind speed; SpecHum–specific humidity; v-wind–meridional wind speed; u-wind–zonal wind speed; Temp–temperature.

5.1. Sensitivity guidance to error covariance weighting

The average values of the forecast sensitivity to the observation-error covariance weight factor (55) for each observed parameter are shown in Figure 2, together with the sensitivity to the background-error covariance weight (57). The comparison is facilitated by the fact that all s-sensitivities have units of J kg−1 since the covariance weight coefficients are non-dimensional scalar parameters. The presence of both negative and positive so-sensitivities indicates that an optimal weighting between the background and observation information may not be explained through a single covariance weight coefficient (e.g. background inflation) and the analysis of each observation type and instrument is necessary to improve the DAS performance.

Figure 2.

Sensitivity (J kg−1) of the 24 h forecast error with respect to the background-error covariance weight factor sb and with respect to the observation-error covariance weight factor equation image associated with each observed parameter. Displayed are average values per assimilation/forecast cycle.

The so-sensitivity values for those observation types whose sensitivity was found to be of increased magnitude are presented in Figure 3. For reference, the sb-sensitivity is also shown in Figure 3. Each sensitivity provides a first-order guidance on the forecast impact as a result of variations in the corresponding weight factor. It is noticed that, as a whole, the observing system information is under-weighted and an increased negative sb-sensitivity indicates that background-error covariance inflation is of potential benefit to the forecasts. Negative so-sensitivities identify those observation types whose error variance inflation (increasing the assigned σo) is of potential benefit to the forecasts, whereas positive so-sensitivities identify those observation types whose error variance deflation (decreasing the assigned σo) is of potential benefit to the forecasts (Daescu and Todling, 2010).

Figure 3.

Sensitivity (J kg−1) of the 24 h forecast error with respect to the observation-error covariance weight factor equation image associated with various observation types (average values per assimilation/forecast cycle for summer and autumn periods): AIRS–radiances from the Atmospheric Infrared Sounder; SSMIS–radiances from the Special Sensor Microwave Imager Sounder; IASI–radiances from the Infrared Atmospheric Sounding Interferometer; AMSU-A–radiances from the Advanced Microwave Sounding Unit-A on the NOAA-15, −16, −18, −19, METOP-A, and Aqua satellites; SSMITPW–Special Sensor Microwave Imager total precipitable water; SHIPSFC–ship and buoy temperatures, winds, specific humidities, and near-surface pressures; LANDSFC–land observations of temperatures, winds, surface pressures, and specific humidities; SATWND–cloud-drift winds; RAOBS–radiosonde temperatures, winds, specific humidities. The forecast sb-sensitivity is also shown.

Figure 4 displays results of OBSI and forecast sensitivity to the error covariance weight factor for the radiosonde observations (RAOBS) distributed between various pressure levels. A large fraction of the RAOBS so-sensitivity is due to the sensitivity to the specific humidity in the low- and mid-tropospheric regions and indicates a suboptimal assimilation of this data type. The OBSI estimates show that the assimilation of each data type is of benefit to the forecasts and the sensitivity guidance is that, on average, improved forecasts may be obtained by increasing the σo assigned to the specific humidities. The increased OBSI and forecast sensitivity which are associated with the specific humidities in the lower troposphere may be explained by the fact that the forecast error is measured in a moist total energy norm and a significant part of the error is comprised of humidity.

Figure 4.

Vertical distribution (hPa) of average values per assimilation/forecast cycle of (a) the observation impact (J kg−1) and (b) sensitivity (J kg−1) to the error covariance weight factor equation image for radiosonde temperatures (T), zonal (u) and meridional (v) winds, and specific humidities (sh) during the summer period. (c, d) are as (a, b), but for the autumn period. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Noticeable in Figure 2 is the change from the summer period to the autumn period in the forecast so-sensitivity to radiances. For each of the atmospheric sounders, the observation count, observation impact, and forecast-error sensitivity to the error covariance weight factor are shown in Figure 5(a, b, c). The sensitivity guidance is that the information provided by AMSU-A is under-weighted in the DAS, whereas inflation of the assigned observation-error variances for IASI and AIRS is of potential benefit to the forecasts. The sensitivity associated with the SSMIS instrument changed from positive values during the summer period to negative values during the autumn period. In the diagnosis studies of Bormann and Bauer (2010) and Bormann et al. (2010, 2011) performed with the European Centre for Medium-Range Weather Forecasts (ECMWF) DAS, it was found that radiances from AMSU-A show little spatial and inter-channel error correlations, and that observation-error correlations of larger magnitude are present in the data provided by the IASI, AIRS, and SSMIS instruments. Their findings also indicate a too-conservative use of AMSU-A.

Figure 5.

Average 0000 UTC (a) observation count/assimilation cycle, (b) forecast impact/observation (J kg−1), and (c) forecast equation image-sensitivity/observation (J kg−1) for the atmospheric sounders. (d, e) Sensitivity (J kg−1) of the 24 h forecast error with respect to the observation-error covariance weight factor equation image of the IASI and AMSU-A instruments for each assimilation/forecast cycle for (d) the summer and (e) the autumn periods.

Given the interaction between various components of the assimilation system, the evaluation of the forecast-error sensitivity with respect to a selected parameter merely provides guidance on the parameter variations that may be used to reduce the forecast errors due to unknown deficiencies in the DAS. In general, the uncertainty in the analysis products increases when radiance observations are dominant (Langland et al., 2008) and, for the analysis/forecast episodes considered in this study, a possible interpretation of the sensitivity guidance for IASI and AIRS is that further inflation of the assigned observation-error variances is necessary to compensate for error correlations that are not represented in the DAS. It is also emphasized that the sensitivity estimates are determined by the DAS configuration, the forecast aspect specification, and the dynamics of the forecast errors. A systematic time-series analysis is necessary to identify tendencies in the forecast-error sensitivity to the information weight. This aspect is illustrated in Figure 5(d, e) where so-sensitivities values obtained for each assimilation/forecast cycle are shown for the AMSU-A and IASI instruments.

5.2. Diagnosis of observation-error variances

Desroziers et al. (2005) have shown that an estimate of the observation-error covariance may be obtained a posteriori from the statistical analysis of the observation-minus-analysis equation image and observation-minus-forecast equation image vectors

equation image(82)

where E[·] denotes the expectation operator, and that estimates equation image of the observation-error variance associated with the data component equation image may be obtained as

equation image(83)

An a posteriori analysis of the NAVDAS-AR products was performed to assess the consistency of the specified observation-error variances for the radiance data, and results obtained during the autumn period are discussed in this section. For each of the atmospheric sounders, the diagnosis estimates equation image derived from (83) were found to be significantly smaller than the corresponding σo values assigned in NAVDAS-AR. An instrument-averaged ratio equation image was obtained as follows: 0.43 for AMSU-A, 0.59 for AIRS, 0.57 for IASI, and 0.35 for SSMIS. Figure 6 displays the diagnosis estimates equation image together with the σo values assigned to each of the AIRS and IASI channels assimilated in NAVDAS-AR during the autumn period. The ratios equation image between the diagnosis estimates and the observation-error standard deviation assigned in NAVDAS-AR to the AMSU-A channels assimilated from NOAA-15, −16, −18, −19, METOP-A, and Aqua satellites are shown in Figure 7. For comparison, the ratios between the diagnosis estimates equation image and the measured instrument noise (NeDT) values provided by Goldberg et al. (2001) are also displayed in Figure 7. It is noticed that the diagnosis estimates for each channel were found to be fairly consistent among various satellites. The ratio equation image is significantly larger for channel 4 than the values obtained for other AMSU-A channels and, given that the NeDT values considered for comparison were published in 2001 and that the diagnostic estimates are valid for the autumn of 2010, the results are consistent with the findings that over this time period the radiometric noise in AMSU-A channel 4 has increased significantly. The estimates equation image were at or below the measured instrument noise for the AMSU-A channels 6, 9, 10, and 11. This aspect is further discussed in the work of Bormann and Bauer (2010) where observation-error estimates provided by various diagnosis methodologies were found to be at or below the measured instrument noise for NOAA-18 AMSU-A channels 5–10.

Figure 6.

The diagnostic estimates to the observation-error standard deviation and the σo values assigned in NAVDAS-AR to the AIRS instrument channels and to the IASI instrument channels assimilated during the autumn period.

Figure 7.

(a) The scaling factor equation image to the AMSU-A observation-error standard deviation evaluated as a ratio between the diagnostic equation image and the equation image specified in NAVDAS-AR. (b) The scaling factor equation image to the AMSU-A observation-error standard deviation evaluated as a ratio between the diagnostic equation image and the measured instrument noise NeDT values from Table 1 in Goldberg et al. (2001).

5.3. The sensitivity guidance for the diagnosis estimates

The estimates equation image may be interpreted as a first iteration of a fixed-point algorithm for tuning the observation-error variances (Desroziers et al., 2005) and their quality may be impaired by various factors including approximations entailed by nonlinearities in the observational operator and misrepresentation of the observation- and background-error correlations (Chapnik et al., 2004; Ménard et al., 2009). An artificial adjustment of various input parameters in the DAS may be used to alleviate deficiencies in the specification of the observation- and background-error covariances. The a posteriori diagnosis is an efficient approach to obtain an internally consistent estimate equation image; however, no guarantee is provided that its implementation in the DAS will have a beneficial forecast impact. On the other hand, the error covariance sensitivity tools are efficient for testing whether a new model equation image will be of potential benefit to a specified forecast aspect, however the estimate equation image is not directly available from the sensitivity analysis. In this context, the combined information derived from the a posteriori diagnosis and from the sensitivity analysis provides grounds to design parameter tuning procedures that are effective in reducing the forecast errors.

An issue to address is whether adjustment of the σo values according to the diagnosis estimates in Figures 6 and 7 is of potential benefit to the forecasts. An a priori guidance is obtained from the equation image-sensitivity analysis which may be used to identify those instrument channels whose reduced (increased) σo values will have a beneficial forecast impact.

5.3.1. The sensitivity guidance for AIRS and IASI

The average values of the forecast equation image-sensitivity per observation for each of the AIRS and IASI channels assimilated in NAVDAS-AR over the autumn period are shown in Figure 8. It is noticed that for each instrument the sensitivity values are closely determined by the channel selection.

Figure 8.

Average values of sensitivity per observation (J kg−1) of the 24 h forecast error with respect to the observation-error variance weight factor equation image for the (a) AIRS and (b) IASI channels assimilated in NAVDAS-AR during the autumn period.

The long-wave CO2 upper temperature-sounding channels of AIRS (channel range 198–251) display negative sensitivities for channels 239–251 and relatively small positive sensitivities for channels 198–232. Among AIRS long-wave CO2 lower temperature-sounding channels 256–355, negative sensitivity values of increased magnitude are noticed for channels 300, 318, 321, and 355, whereas positive values are associated with channels 305 and 309. AIRS long-wave window channels 362–672 and 950 exhibit negative sensitivities and with values of increased magnitude for channels 362, 375, and 453. Noticeable in Figure 8 are the positive sensitivities of increased magnitude that are associated with AIRS short-wave temperature-sounding channels 1995 and 2084, and this is an indication that reducing the assigned observation-error variances for these two channels to the diagnosis estimates equation image is of potential benefit to the forecasts.

The long-wave CO2 upper temperature-sounding channels of IASI (channel range 122–249) display small sensitivity values for channels 122–236 and negative values of increased magnitude for channels 239, 246, and 249. Most of the long-wave CO2 lower-peaking temperature-sounding channels 252–379 of IASI exhibit negative sensitivities of increased magnitude.

For the long-wave window channels of AIRS and the majority of the long-wave CO2 lower temperature-sounding channels of both AIRS and IASI, the sensitivity analysis guidance is that further inflation of the assigned observation-error variances is of potential benefit to the forecasts and thus a equation image-specification according to the diagnosis estimates equation image may have a detrimental impact on the DAS performance. It is also noticed that the diagnostic estimates of Bormann et al. (2010) in the ECMWF system indicate that increased interchannel error correlations may be present in the long-wave CO2 lower temperature-sounding channels and the long-wave window channels of AIRS and IASI, whereas in their study small or no correlations were found for the mid-tropospheric to stratospheric long-wave temperature-sounding channels of these instruments.

5.3.2. The sensitivity guidance for AMSU-A

The average values of forecast equation image-sensitivity per observation for each of the AMSU-A channels assimilated in NAVDAS-AR over the autumn period are shown in Figure 9. Positive values are associated with channels 5–8 and indicate that reducing the σo values assigned to these channels is of potential benefit to the forecasts. An increased magnitude is noticed for the forecast equation image-sensitivity to AMSU-A channel 6. The sensitivity analysis distinguishes AMSU-A channel 4 as an instrument channel whose negative equation image-sensitivity of increased magnitude indicates that further inflation of the assigned observation-error variance is necessary to improve its performance in the DAS. Negative sensitivity values of relatively small magnitude were obtained for AMSU-A channels 9–14.

Figure 9.

As Figure 8, but for the AMSU-A channels. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

5.3.3. A priori first-order impact estimates

An additional application of the equation image-derivative information is to provide an a priori assessment of the potential benefit that may be achieved from adjusting the observation-error variances to the diagnosis estimates equation image. The first-order approximation (72) may be used to obtain a measure of the additional forecast-error reduction that may be gained from tuning the observation-error variances associated with a selected set of instrument channels while maintaining the status quo for all other input components in the DAS,

equation image(84)

The analysis of each instrument channel is necessary to optimize the performance of covariance tuning procedures and insight on the channel selection may be obtained from the sensitivity guidance. This aspect is illustrated in Figure 10 where the OBSI assessment for AMSU-A in the reference NAVDAS-AR configuration is shown together with the first-order impact estimates (72) to the additional forecast-error reduction that may be achieved by adjusting the observation-error variance associated with the AMSU-A channels 5–8 to the diagnosis estimates equation image. A ratio in the range 12–22% was obtained for various satellites, as shown in Figure 10(b). For comparison, the first-order estimates to the additional forecast-error reduction that may be achieved by adjusting the observation-error variances associated with all AMSU-A channels to the diagnosis estimates equation image are also shown in Figure 10. In this case, the first-order estimates incorporate a detrimental forecast impact from adjusting the observation-error variances associated with the AMSU-A channels 4 and 9–14. Consequently, the estimates to the additional reduction in the forecast error display a lower magnitude in the range 3.5–12%.

Figure 10.

(a) The observation impact (J kg−1) associated with AMSU-A in the reference NAVDAS-AR configuration (OBSI) and the first-order estimates to the additional forecast-error reduction when tuning of the observation-error variances to the diagnostic estimates equation image, is applied to channels 5–8 only (OBSI+ ch5-8), and to all channels (OBSI+ all ch). (b) The estimates to the additional forecast-error reduction expressed as a percentage of the observation impact in the reference NAVDAS-AR configuration. Average values per assimilation/forecast cycle are shown for the autumn period. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

For the short-wave temperature-sounding channels 1995 and 2084 of AIRS, the average values of OBSI per assimilation/forecast cycle during the autumn period were –0.010 and –0.012 J kg−1, respectively. To a first-order approximation, the additional OBSI to the forecast-error reduction that may be gained from adjusting the observation-error variances to the diagnostic values equation image was estimated to be 9.8% for AIRS channel 1995 and 8.9% for AIRS channel 2084.

Caution must be exercised in the interpretation of these results given the fact that OBSI estimates are based on the second-order measure (27)–(28), whereas the assessment of the additional forecast impact from tuning the observation-error variances is of lower accuracy since it is based on the first-order measure (72). It is noticed that the approximation (72) relies on the sensitivities (38) which are evaluated in the reference DAS configuration (equation image) such that the guidance on the forecast impact from tuning the observation-error variances is obtained prior to the assimilation experiment with the DAS configuration equation image. A feasible approach to obtain a second-order approximation may be considered only after the assimilation experiment is performed by incorporating information derived from an additional adjoint-DAS evaluation of the sensitivity of the forecast equation image to the observation-error variance specification equation image. The use of various gradient trajectories to increase the order of accuracy may be interpreted as a numerical integration (quadrature) approximation (Daescu and Todling, 2009) and the practical applicability of this methodology to assess the impact of covariance parameters remains to be further investigated.

A limitation of the adjoint-DAS R- and B-sensitivity and impact estimation is that this approach is only valid for a specific forecast aspect and does not capture the long-term impact of modifications in the error covariance models, i.e. the forecast impact propagation through various data assimilation/forecast cycles. Therefore, the outcomes of the error covariance sensitivity and impact guidance must be cautiously interpreted and a judicious validation through OSEs is necessary to assess the performance of new error covariance models. Nevertheless, valuable insight is gained by monitoring the forecast sensitivity to R and B parameters along with OBSI and diagnosis estimates.

6. Conclusions

This article provides the theoretical basis to the adjoint estimation of the forecast sensitivity to the specification of the observation- and background-error covariances in a strong-constraint 4D-Var DAS with a single outer-loop iteration. The practical ability to obtain sensitivity information with respect to R- and B-parameters was presented with the adjoint version of NAVDAS-AR developed at the Naval Research Laboratory. An additional contribution of this work is that it provides a link between various methodologies to analyze the DAS performance: observation sensitivity and impact assessment, error covariance sensitivity, and a posteriori diagnosis. In NAVDAS-AR, the evaluation of R- and B-sensitivity relies on the adjoint-DAS software tools developed for observation sensitivity and impact analysis, and these computations may be performed simultaneously to obtain complementary information on the use of observations in the DAS. The OBSI measures the performance of each observing system component on reducing the forecast errors, as determined by the current error covariance specification (R,B). The R- and B-sensitivity provide guidance on the adjustments in the error covariance parameters that are necessary to improve the forecasts, given the observing system configuration. The derivative information identifies the steepest descent direction in the parameter space to reduce the forecast errors, however an optimal specification of the parameter values may not be inferred from the sensitivity analysis. The information extracted from error covariance diagnosis and sensitivity estimates allows the investigation of new parameter tuning procedures that combine the strengths of each of these methodologies and alleviate their shortcomings. At the time of this study, research is in progress to incorporate the sensitivity guidance in the design of OSEs and the outcomes of the experiments performed with NAVDAS-AR will be provided in our future work.

The observation sensitivity vector is a key ingredient to R- and B-sensitivity and impact estimation. Techniques to evaluate the observation sensitivity in an Ensemble Kalman Filter DAS have been developed (Liu et al., 2009) such that the expressions for R- and B-sensitivity and impact estimation derived in this study in the adjoint-DAS 4D-Var framework may also be evaluated in ensemble-based DASs. Computationally efficient estimates of the forecast performance of new covariance models may be obtained by exploiting the mathematical properties of the R- and B-sensitivity matrices. An extended range of applications will be investigated in our further work to include the evaluation of the forecast sensitivity to observation- and background-error correlations, the assessment of observation-error correlation models derived from diagnosis estimates, and guidance to the specification of a flow-dependent background-error covariance model in a hybrid variational–ensemble DAS. NAVDAS-AR allows the implementation of a weak-constraint 4D-Var analysis scheme, and a theoretical framework to forecast sensitivity and impact estimation for model-error covariance parameters needs to be formulated.

Acknowledgements

The work of D. N. Daescu was supported by the Naval Research Laboratory Atmospheric Effects, Analysis, and Prediction BAA #75-09-01 under award N00173-10-1-G032 and by the National Science Foundation under award DMS-0914937. Support for the second author from the sponsor ONR PE-0602435N is gratefully acknowledged.

Appendix

The equation (57) of the forecast sensitivity to the background-error covariance coefficient sb in the parametric representation (54) may be also derived by replacing δB = δsbB in the equation of the first-order forecast variation (47)

equation image(85)

The identity

equation image(86)

is established as detailed below.

equation image

The sensitivity (57) is obtained by replacing (86) in (85).

Ancillary