SEARCH

SEARCH BY CITATION

Keywords:

  • 3D-Var;
  • balance constraints;
  • forecast error covariances

Abstract

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methodology and validation
  5. 3. Experiments with 3D-Var
  6. 4. Discussion and conclusions
  7. Acknowledgements
  8. Appendix: Tangent Linear Equations
  9. References

Mass–wind and vorticity–divergence balance constraints based on the linearized Charney and quasi-geostrophic omega equations, respectively, are assessed in a developmental version of the global, three-dimensional variational data-assimilation system at Environment Canada. Unlike traditional balance constraints, which are averaged in time, the new constraints are flow-dependent and reflect a more complete set of dynamics. Single observation experiments demonstrate that the new covariance model leads to asymmetrical increments that are qualitatively aligned with the instantaneous background wind field. Data-assimilation experiments using real observations are performed for a period of five weeks during two different seasons, employing the control and experimental constraints. Subsequent forecast verification against radiosondes shows a definite benefit of the new covariances in the Tropics; however, the impact in the Extratropics is neutral or slightly negative. Verifications against analysis show virtually no change in the troposphere; however, a significant improvement is observed in the stratosphere at all lead times. Compared with the Charney mass–wind balance, the contribution of the quasi-geostrophic omega constraint is rather minimal, at least in its current adiabatic form. The new balance scheme requires a considerable amount of computational time in the context of our 3D-Var system, although the relative cost in a 4D-Var setting may be far less significant. Moreover, the present experiments are useful in elucidating several important aspects of covariance modelling, particularly the dependence of balance dynamics on spatial scale. © 2012 Crown in the right of Canada. Published by John Wiley & Sons Ltd.


1. Introduction

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methodology and validation
  5. 3. Experiments with 3D-Var
  6. 4. Discussion and conclusions
  7. Acknowledgements
  8. Appendix: Tangent Linear Equations
  9. References

Specification of background-error cross-covariances in the context of numerical weather prediction (NWP) continues to be an area of active research and debate, despite numerous advances in data-assimilation methodologies. Such covariances serve as constraints on the model atmosphere by helping to maintain balance relationships inherent in atmospheric motions and by spreading the information from observations in a physically consistent manner. This is particularly important in view of the relative paucity of observations of some state variables (e.g. winds) compared with others (e.g. temperature, via radiances from satellites). Constraints are also necessary to minimize dynamic adjustment through gravity waves, which are often incompletely or poorly observed (Alexander et al., 2010) and therefore tend to cause a deterioration of subsequent forecasts and analyses.

Besides its importance for tropospheric dynamics in the NWP context, balance has a significant influence on stratospheric processes, such as the transport of pollutants and other atmospheric constituents via the middle atmospheric residual circulation (Bregman et al., 2006). Assimilation systems currently in operational use have a tendency to cause excessive mixing in the lower stratosphere (Schoeberl et al., 2003), a general problem related to persistent underestimation of the ‘age of air’ (see Hall and Plumb, 1994 for the definition) and an overestimation of the strength of the Brewer–Dobson circulation in global analyses (Monge-Sanz et al., 2007), which hampers long-term chemical transport studies. At the European Centre for Medium-Range Weather Forecasts (ECMWF) it was found that refinements in the treatment of balance were responsible, at least in part, for an improvement in the age-of-air diagnostic relative to observations (B. Monge-Sanz, personal communication).

Covariances of the background error may be modelled explicitly or derived statistically. However, in either case the physical motivation is the existence of a balance between certain atmospheric fields, based on the equations of fluid motion. A number of such diagnostic relations are available, for example geostrophic balance (and its generalizations such as linear balance and the Charney nonlinear balance) and hydrostatic balance. However, the most appropriate application of such balance constraints for a practical and effective assimilation scheme in the NWP context is not at all obvious. For a more in-depth discussion, the reader is referred to Bannister (2008a, 2008b), where the state of covariance modelling was recently reviewed.

Parrish and Derber (1992) introduced a technique designed for atmospheric data-assimilation systems whereby static covariances reflecting the linear balance are computed by regression from a series of differences between forecasts and corresponding analyses or between forecasts with different lead times (the so-called NMC (National Meteorological Center) method). The resulting multivariate statistics typically retain latitudinal dependence, can be pre-computed for each month of the year and may be specified in spectral space, which has certain computational advantages (Rabier et al., 1998; Derber and Bouttier, 1999). The technique was incorporated in variational assimilation systems at a number of operational NWP centres, including ECMWF (Courtier et al., 1998) and Environment Canada (EC: Gauthier et al., 1999). It is noteworthy that both three- and four-dimensional (3D-Var and 4D-Var, respectively) schemes benefit from such dynamical constraints. Although currently the balance operators in the EC operational scheme rely on the NMC method, EC and other NWP centres are increasingly moving toward hybrid systems, which utilize time-dependent ensembles of forecasts or analyses to estimate the background-error term (Buehner et al., 2010a, 2010b; Bonavita et al., 2011; Raynaud et al., 2011) while still within the Derber and Bouttier (1999) formalism.

A drawback of the NMC approach is the lack of flow dependence in the covariance structures (unless this feature is introduced through flow-dependent coordinates or some other methodology, such as in Fillion et al., 2007 or Kleist et al., 2009). Fisher (2003) showed that a covariance model based on the linearized Charney and quasi-geostrophic (QG) omega equations (Charney, 1963; Hoskins et al., 1978) can produce increments with shape and alignment consistent with the instantaneous mean flow. This is particularly apparent in regions of strong wind curvature and occurs due to the background-dependent terms in the equations. A wavelet-based version of this formulation was subsequently adopted operationally at ECMWF (Fisher, 2004; IFS, 2006).

In the present study we aim to address two key questions. Firstly, what benefit, if any, can be provided by a constraint that is better in theory but highly simplified in practice? Secondly, do such benefits outweigh the increased complexity of the balance scheme? The equations as introduced by Fisher (2003) are simplified in several respects, chiefly in that corrections due to the transformation between pressure and model coordinates are neglected (see also Lindskog et al., 2007). We have followed a somewhat similar approach and introduced the equations as physical-space constraints in a global 3D-Var ‘first guess at appropriate time’ (FGAT) data-assimilation scheme derived from the EC operational global deterministic prediction system (GDPS). Significant effort is made to document the effects of the new covariances, applied individually and together, on resulting analyses and forecasts. Unlike previous publications, we present comparisons of objective forecast scores for long-term, quasi-operational assimilation experiments, isolating the impact of the new flow-dependent constraints. Moreover, we discuss the undesirable effects of unbalanced motion at large horizontal scales, which may be relevant for other assimilation systems whether or not they utilize flow-dependent constraints.

This article is organized as follows. The introduction of balance constraints in an incremental variational assimilation system, including the change of variable from total to unbalanced control variables, is reviewed in section 2. The same section also describes the numerical treatment of the linearized balance equations as well as validation of the solutions. Section 3 is devoted to a presentation of our assimilation experiments using the new constraints as well as forecast verifications against both radiosondes and analyses. The relative cost in terms of computational resources associated with different constraints is also indicated. A more in-depth discussion of the results and some concluding remarks are contained in section 4. The Appendix provides the explicit tangent linear equations used in our study and a few notes regarding their derivation and implementation.

2. Methodology and validation

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methodology and validation
  5. 3. Experiments with 3D-Var
  6. 4. Discussion and conclusions
  7. Acknowledgements
  8. Appendix: Tangent Linear Equations
  9. References

2.1. Balance in incremental 3D-Var

We begin with a description of the general approach of introducing new balance constraints into the assimilation scheme. The 3D-Var incremental cost function is

  • equation image(1)

where δx is the analysis increment, xref is the reference or background state, y′′ is the innovation vector, B and R are the background- and observation-error matrices respectively and H is the linearized forward operator. The vector δx is defined as

  • equation image(2)

where δψ, δχ, (δT,δps) and δlnq are the incremental control variable vectors: total stream function, total velocity potential, total temperature (and surface pressure) and logarithm of total specific humidity, respectively. We note that the treatment of the humidity variable was not addressed in this study. In practice, a change of control variables is performed such that B is assumed to be diagonal in control-variable space, and the transformation is designed to make this assumption plausible. For further details we refer the reader to Gauthier et al. (2007) and Laroche et al. (2007).

To introduce a new balance operator, the change of variable includes a transformation from total to unbalanced fields. The transformation has the form

  • equation image(3)

where

  • equation image(4)

is the unbalanced component of δx, K satisfies

  • equation image(5)

and Bu is the error correlation matrix associated with δxu. By definition, all the cross-covariances in Bu are zero, although the matrix still contains spatial correlations. K can be written as

  • equation image(6)

where E and N are independent of xref (Gauthier et al., 1999). Although E and N are not invertible, the inverse of K exists.

With a nonlinear balance, the change of variables from δxu to δx becomes a function of the reference state and can no longer be written as a matrix such as E or N. The change from balanced to unbalanced variables,

  • equation image(7)

will involve the Charney and QG omega equations, and may be written as

  • equation image(8)
  • equation image(9)
  • equation image(10)
  • equation image(11)

where the subscript ‘b’ denotes balanced quantities. Taking the variation yields

  • equation image(12)

where

  • equation image(13)

Here,

  • equation image(14)

Note that the inverse of K−1 cannot be written down so easily as in the linear case.

The operators E2, N2 and E3, explicit expressions for which are presented in the Appendix, are functions of the reference state and their application involves a non-trivial numerical scheme that required prior validation. This is in contrast to static covariances, such as those based on regression, which can be computed in advance and applied as straightforward matrix operations. Moreover, the above framework, in which the balanced fields δTb, δps,b and δχb are formally functions of linear stream-function perturbations δψ, poses certain challenges that merit discussion.

2.2. Development of solution procedure

The balance constraints used in this study were obtained by linearizing the Charney, hydrostatic, QG omega and continuity equations in pressure coordinates (Holton, 1992), followed by a transformation to η, the native vertical coordinate of the forecast model. The associated tangent linear (TL) equations are given in the Appendix, along with a few remarks regarding the derivation and relevant approximations.

A solution procedure was implemented in FORTRAN, using spectral harmonics to evaluate horizontal gradients, which is particularly convenient for the Laplacian operator and its inverse. All equations were solved globally and no additional procedures were applied to impose spatial localization (unlike Gauthier et al., 1999, where a local form of linear balance was used). Vertical derivatives were computed via a pressure-weighted three-point centred difference formula except in the QG omega equation, which was solved following the methodology proposed in Fillion et al. (2005), i.e. separation of variables followed by finite-element decomposition of the vertical structure. The resulting eigenvalue problem was handled using the LAPACK library (Anderson et al., 1999).

The forecast model used in this study was a stratospheric version of the Global Environmental Multiscale (GEM) model, with a lid at 0.1 hPa and otherwise a configuration similar to that described in Ménard et al. (2007) and de Grandpré et al. (2009). This (hydrostatic) version of the model, which will be referred to as GEM–Strato, uses a horizontal discretization grid of 240 × 120 with 80 vertical levels, which are terrain-following but relax to isobaric coordinates with altitude. GEM–Strato should be differentiated from the current operational model at EC, which also has a lid at 0.1 hPa but operates at a horizontal resolution of 800 × 600 (Charron et al., 2011). The 3D-Var FGAT assimilation system used in the present study was the same as the GDPS at EC in 2003–2004 when this research was initiated, with some modifications to account for a higher lid (at that time the lid in the operational system was at 10 hPa; see Gauthier et al., 1999).

The EC operational GDPS has undergone a number of changes in recent years, particularly in the formulation of the balance operators, progression from 3D-Var to 4D-Var and addition of new observations. Much investigation has been focused on the specification of covariances based on ensemble forecasts and on hybrid methods utilizing both forecast ensembles and statistical regression (e.g. Buehner, 2005; Buehner et al., 2010a, 2010b), however such schemes have not yet been introduced and the mass–wind and vorticity–divergence balances used operationally are essentially computed as time-independent covariances based on forecast differences. The latter balance, reflecting the process of boundary-layer Ekman pumping (Polavarapu, 1995), is only applied in the lowest few levels of the domain, whereas throughout the free atmosphere the velocity potential and therefore the divergence is unconstrained.

Before implementation of the new constraints in the assimilation scheme, the solution routines were extensively investigated in an offline setting. Here, for brevity, the control mass–wind constraint will be referred to by the acronym SB (statistical balance) while the experimental constraints will be called LB (linear balance), CB (Charney balance) and QG (QG omega equation). Explicitly, the equations used for each constraint are as follows:

  • CB: (A1), (A2) and (A5);

  • LB: as CB, omitting terms which involve uref,vref;

  • QG: (A3) and (A4).

It should be noted that the SB statistics are based on a variation of linear balance called local balance, which limits the spatial extent over which an observation will have a noticeable influence (Gauthier et al., 1999). Thus, cases SB and LB lead to similar, but not identical, results.

A correct characterization of the temperature background error is an important aspect of the assimilation system. Standard deviations of the balanced temperature δTb were computed using constraints SB and CB (see (A1)–(A2)) based on an ensemble of 60 stream-function samples δψ and are shown in Figure 1. The samples correspond to differences between 24 h and 48 h forecasts (valid at the same time), launched every 12 h from a previously executed month-long assimilation cycle. Thus in the SB case the calculation simply reflects the NMC method, while in case CB the computation indicates the minimal set of dynamical features that can be captured diagnostically by the new constraint. Panels (b) and (c) should be compared with the standard deviation of the actual temperature samples (a), which here serves to estimate the variability of the true atmosphere.

thumbnail image

Figure 1. (a) Standard deviation of temperature samples δT [K] based on an ensemble of 24–48 h forecast differences. Forecasts were launched every 12 h from a January 2007 data-assimilation cycle, for a total of 60 ensemble members. (b) and (c) Standard deviation of balanced temperature δTb [K] computed using the (b) SB and (c) CB constraints, based on stream-function samples δψ from the same ensemble as in (a). Summation was over longitude and over all samples. η is the model vertical coordinate.

Download figure to PowerPoint

It is clear that neither balance constraint captures the dynamics of the Tropics or the stratosphere very well. Indeed, since both methods are based essentially on geostrophy, the resulting variances necessarily decrease and vanish approaching the Equator. In the extratropical troposphere, however, constraint CB seems to resolve the arch-like shape of the true forecast-error structure (Figure 1(a)) somewhat better than SB, which has a beneficial influence on increments. Otherwise, the error distribution in panels (b) and (c) is quite similar, suggesting that the impact of the CB operator on time-averaged statistics will not be dramatic.

Although the error variance in Figure 1(c) is physically plausible, instantaneous solutions δTb were found to exhibit unrealistic artefacts at large scales due to the unbalanced component of δψ, particularly evident in the zonal-mean structure. This spurious mode, which consists of broad extratropical anomalies alternating in sign with height, is actually rather ubiquitous and can be discerned in stream-function and temperature increments from standard 3D-Var assimilation cycles that rely on static covariances, as well as in differences of successive states from free model simulations, including 6, 12, 18 and 24 h differences (not shown). Due to the presence of the large-amplitude feature in the winter-hemisphere stratosphere, as well as the convergence of longitudes, zonal derivatives in the Charney term of (A1) appear to be overestimated near the North Pole.

It should be stressed that the unbalanced mode is manifested strongly not in the stream function itself but in the fields related to its adjustment, such as increments and temporal model differences, as might be expected (Lorenc et al., 2003). We obtained very similar results with free simulations of the Canadian Middle Atmosphere Model (CMAM: Beagley et al., 1997), a climate-chemistry model developed jointly by EC and the University of Toronto. The CMAM is spectral and has a much higher lid than GEM–Strato, which suggests that the anomalous δTb structure is a generic issue.

Because solution of the full Charney and hydrostatic balance relations ((11.15) and (6.2) in Holton, 1992) using our methodology for a given model state recovers the corresponding model temperature T with no difficulty, we conclude that the nonlinearized Charney and hydrostatic balances could serve as excellent constraints for the purposes of variational assimilation. However, since most operational systems are based on the incremental formulation (Courtier et al., 1994), the appearance of a strong unbalanced mode in the linearized solutions could be problematic for any scheme relying on a separation of the increment into balanced and unbalanced components. In the long term, the problem may require a fundamentally different approach to covariance modelling, as will be discussed further in section 4.

In order to make progress within the context of the current implementation of 3D-Var, we adopted two relatively simple procedures that allowed us to obtain physically sensible instantaneous solutions of δTb. The spurious mode is strongest at the largest horizontal scales, which is consistent with the arguments of Lorenc et al. (2003) and Wlasak et al. (2006), based on geostrophic-adjustment theory. Therefore, when solving for δΦb in the LB or CB case, the right-hand side of (A1) is spectrally filtered, setting the lowest five spectral coefficients (both the zonal and total wave number) to zero. Thus, the resulting solution contains little information on planetary scales; however, it is much less contaminated by the unbalanced component of δψ. A similar technique was employed in Bannister (2009). For our purposes, we determined the wave-number cut-off value empirically, by examining a number of selected δTb fields computed from forecast differences δψ. Additionally, the right-hand side of (A1) is smoothed vertically using straightforward nearest-neighbour averaging in order to reduce sharp gradients further.

2.3. Validation of solution procedure

Much of the validation of our balance equations was performed in the context of model 6 h differences. It is not suggested that forecast-error statistics should be based on these differences in an operational setting. Nevertheless, if we view the evolution of the balanced analyzed state as being analogous to the geostrophic adjustment of the true atmosphere (Lorenc et al., 2003), then applying our balance relations to successive 6 h model differences is likely to indicate the kinds of impacts that the constraints would induce in an actual assimilation cycle. The procedure is convenient, since the model provides dynamically consistent, regularly gridded fields of all the desired fields. Although not discussed here, static covariances derived from 6 h differences also offer a practical, efficient alternative to the NMC method (Polavarapu et al., 2005; Jackson et al., 2008) when ensembles of 24–48 h forecasts are not available.

Figures 2 and 3 show 6 h differences in model fields along with the corresponding balanced fields computed using constraint CB at a randomly chosen time-step during a free simulation of GEM–Strato. The instantaneous model state was used as the background while the model stream function 6 h difference served as δψ. As Figure 2(a) and (b) demonstrates, the synoptic, horizontal structure of the model temperature is reproduced quite well by the balanced field. The degree of accuracy varies with vertical level, however in general the qualitative spatial patterns are captured. This suggests both that the dominant, synoptic-scale patterns of dynamic adjustment are largely balanced and that constraint CB captures much of this balance.

thumbnail image

Figure 2. (a) and (b) 500 hPa sections and (c), (d) and (e) zonal means of (a) and (c) model temperature 6 h difference δT [K] and (b) and (d) corresponding balanced temperature δTb [K] computed from (A1) and (A2) based on the model stream function 6 h difference δψ and the model state at a randomly chosen moment during a free simulation of GEM–Strato under January conditions. (e) shows δTb computed from (A2) only, using the model geopotential 6 h difference. η is the model vertical coordinate.

Download figure to PowerPoint

thumbnail image

Figure 3. (a) Model surface pressure 6 h difference δps corresponding to the fields in Figure 2. (b) Corresponding balanced surface pressure δps,b computed from (A5).

Download figure to PowerPoint

Agreement between δT and δTb in the zonal mean, on the other hand, is rather poor, as shown by Figure 2(c) and (d), despite the application of the spectral filter and vertical smoothing outlined above. The alternating positive/negative anomalies in the temperature vertical structure in high latitudes, visible particularly in the stratosphere, are similar to deficiencies in analyses and increments reported previously concerning other datasets (Uppala et al., 2005; Tremolet, 2007; Lindskog et al., 2009). By contrast, these authors used time-invariant, regression-based constraints, which indicates that the issue is generic and not related to flow dependence.

The appearance of such vertical oscillations in the upper troposphere and stratosphere is a ‘long-standing problem in satellite data assimilation’ and has been attributed to ‘large and systematic discrepancies between the forecast model and the radiance information’ (Dee, 2005; Dee and Uppala, 2009); however, the exact nature of these model biases has not been explained. Since the anomalies are strongest in the zonal mean (i.e. on the longest zonal length-scales), we suggest that the issue is at least partly due to the planetary-scale (i.e. most unbalanced) components of δψ. As noted in section 2.2, if the stream-function increment contains unbalanced components of significant amplitude then any resulting balanced temperature increments are likely to be contaminated as well and the unbalanced signal will then appear in subsequent analyses.

Although the vertical derivative in the hydrostatic equation (A2) undoubtedly amplifies existing gradients, we have found that the associated finite-difference scheme is not in itself responsible for the anomalous large-scale vertical structure in δTb. Indeed, the precursors of the anomalous temperature features already appear in the geopotential field resulting from horizontal balance (not shown). Furthermore, if we provide model 6 h differences of the geopotential to our hydrostatic operator then it recovers the model 6 h temperature difference with reasonable accuracy, including the longest zonal scales, as may be seen in Figure 2(e). The so-called computational mode (Hollingsworth, 1995) can be ruled out as the source of the anomalies, since their vertical scale is considerably deeper than the thickness of two adjacent model layers. Despite the shortcomings of the horizontal operator, the vertical structure shown is a significant improvement over solutions without the benefit of spectral filtering and smoothing (which exhibit unacceptably large amplitudes) and overall the scheme was deemed to be adequate for data-assimilation experiments.

Figure 3 illustrates the model surface pressure 6 h difference along with the numerical solution to (A5) for the same background and δψ as in Figure 2. On synoptic scales, the balanced field contains all the same features as the model field, which is further evidence that the adopted scheme captures a significant amount of the dynamical variability. The amplitude of δps,b tends to be somewhat smaller than that of δps; however, this is not unreasonable, since the model field is the sum of a balanced plus an unbalanced component. A small-amplitude wavelike anomaly with zonal wave number 2 is visible in the Tropics in Figure 3(a) but is missing from the balanced solution in Figure 3(b), presumably due to the application of the low-band spectral filter.

Validation of the QG omega solution using model fields is more difficult because, unlike temperature, vertical motion is mostly unbalanced. It is, in fact, dominated by gravity waves, which are precisely the modes that we hope to constrain utilizing the QG approximation. A generalized form of the nonlinear QG omega equation has proven to be a useful offline diagnostic of localized balanced vertical motion as well as a viable initialization scheme in regional forecast systems (see for example Tsou et al. (1987), Caron et al. (2007a, 2007b), Pagé et al. (2007) and references therein). However, given that formally the QG approximation is only valid away from the Tropics, relatively few authors have investigated its utility in the global context.

Lacking global fields of balanced, adiabatic ascent and descent for comparison purposes, the tangent linear QG omega equation (A3) was solved specifying a simple background state and an idealized, localized perturbation in vorticity δζ. Specifically, the reference state consisted of zonally averaged model fields at a randomly selected time step during a January forecast, while the perturbation was a simple Gaussian-shaped anomaly centred at 46°N, 74°W, 493 hPa. Horizontal velocities associated with the vorticity perturbation are shown in Figure 4(a), while the resulting QG omega solution is shown in Figure 4(b) and (c). The response in vertical motion is localized as expected, with rising (sinking) motion to the east (west) of the vorticity anomaly, respectively. Taking into account the predominantly westerly background winds at this spatial location (not shown), the response is consistent with the expected motion for the QG omega approximation (Bluestein, 1992).

thumbnail image

Figure 4. (a) Horizontal sections at 520 hPa of the (dark contours) zonal and (light contours) meridional velocity anomaly [m s−1] associated with an imposed vorticity perturbation at 46°N, 74°W, 493 hPa, as described in the text. Continuous and dashed curves correspond to positive and negative values, respectively. (b) Horizontal section at 520 hPa and (c) vertical section along 46°N of the corresponding QG omega linear solution [mPa s−1] to (A3) for a zonal-mean background state. (d) QG omega solution at (light curves) 493 hPa and (dark curves) 520 hPa along 71°W, one of the two zonal maxima of the solution, where the continuous (dashed) curves were obtained with (without) the η correction terms in (A3). The dashed lines in (b) and (c) show the longitude and altitudes corresponding to the curves in (d).

Download figure to PowerPoint

An example of the effect of the η-coordinate correction terms (terms involving advection of ps) in (A3) is presented in Figure 4(d), where we have plotted the QG omega solution δωb with (continuous curves) and without (dashed curves) the η corrections. The solution is plotted along 71°W, where δωb attains its largest (negative) values, at two selected pressure levels: (dark curves) 493 and (light curves) 520 hPa. While including the correction terms results in a relatively smooth solution, their neglect brings about some spurious oscillation between 30°N and 46°N, which leads us to conclude that these terms should be retained in the final formulation. Due to the time constraints of this study, it was not feasible to implement and test all the corrections that theory suggests, and (A1)–(A5) are certainly not claimed to be optimal in that regard.

3. Experiments with 3D-Var

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methodology and validation
  5. 3. Experiments with 3D-Var
  6. 4. Discussion and conclusions
  7. Acknowledgements
  8. Appendix: Tangent Linear Equations
  9. References

3.1. Single-observation experiments

The preceding assessment indicates the quality of balanced fields based on perturbations of the stream function, computed with the TL equations in a simplified, linear setting. However, in a variational assimilation scheme there will be a feedback on the stream-function variable through the adjoint model. Further nonlinear feedbacks are to be expected from the interaction between forecast, analysis and observations through the assimilation algorithm.

Adjoint equations corresponding to (A1)–(A5) were introduced into the 3D-Var scheme, including the spectral filter and vertical smoothing described in the previous section. Internal consistency of the new subroutines was verified using the adjoint test and gradient test, both individually and in series. Subsequently, assimilation experiments were performed using GEM–Strato as the forecast model, both for individual dates using synthetic observations and for five-week periods utilizing all the standard meteorological observation types available in the EC operational GDPS in 2004, including conventional observations (e.g. radiosondes, aircraft, surface observations), satellite radiances and atmospheric motion vectors from geostationary satellites. The long-term assimilation cycles were executed under winter and autumn conditions in order to assess the impact of the balance constraints during different seasons.

The control assimilation configuration SB employed static covariances very similar to those implemented in the operational system at the time of this study. Experiments were performed using the mass–wind constraints LB and CB, defined previously, as well as the following combined constraints:

  • SBQG: both SB and QG;

  • CBQG: both CB and QG.

In cases LB and CB, our mass–wind balance entirely replaces the control background-error statistics SB; however, the approach with the balanced velocity potential is somewhat different. In cases SBQG and CBQG, the QG omega and incompressibility operators are applied in the free atmosphere, while in the planetary boundary layer a transition is smoothly made to the standard Ekman-balance covariances. Thus, near the lower boundary where the QG omega relation is less likely to hold all experimental cycles employ the same vorticity–divergence constraint as the control.

In order to examine the flow dependence induced by the Charney balance, several single-observation experiments were performed with 3D-Var, imposing a synthetic observation in the geopotential height field Z at 80°W, 50°N, 300 hPa. The resulting increments corresponding to balance constraints SB, LB and CB are plotted as contours in Figure 5 while shading shows the background geopotential field, which serves as a proxy for the shape of the tropospheric westerly jet. The observation was deliberately placed in a region of strong jet curvature in order to maximize the flow-dependent features of the increments. The increments in panels (a) and (b) are zonally symmetric and do not seem to be influenced by the background flow. By contrast, panel (c) suggests that the CB constraint deforms the increment in a way that is consistent with the mean winds. The response is somewhat more intensified in case LB than in case SB, and even more so in case CB. The asymmetric nature of the increment in panel (c) is to be expected on physical grounds, and was demonstrated in Fisher (2003). The vertical structure of the CB increment (not shown) is similar qualitatively to cases SB and LB, but shows a more intense, localized response.

thumbnail image

Figure 5. Contours show the temperature response [10−2K] at 279 hPa due to a single, imposed geopotential height observation at (80°W, 50°N, 300 hPa) on 30 August 2003 using 3D-Var with (a) SB, (b) LB and (c) CB constraints. The shading shows the background geopotential height field (identical in all plots), with decreasing values toward the north, as a proxy for the mean wind structure.

Download figure to PowerPoint

Because the Charney and QG omega equations reflect the same order of accuracy in balance theory, it is most appropriate to use both constraints together. Figure 6 demonstrates the effect of the CBQG constraint given a single temperature observation specified at the same location as above. Contours show the corresponding zonal velocity increments δu for cases SB, LB and CBQG, while shading shows the background geopotential height. As before, panels (a) and (b) are similar, while in panel (c) the increment structure is modified and follows the ambient wind. Since the Charney balance has no direct influence on the velocity, the effect is primarily due to the QG omega balance. Increments of the meridional velocity (not shown) exhibit similar features, in particular a stronger zonal gradient near the location of the observation.

thumbnail image

Figure 6. Contours show the zonal velocity response [m s−1] at 279 hPa due to a single, imposed temperature observation at (80°W, 50°N, 300 hPa) on 30 August 2003 using 3D-Var with (a) SB, (b) LB and (c) CBQG constraints. The shading shows the background geopotential height field, which is the same as in Figure 5.

Download figure to PowerPoint

3.2. Long-term assimilation experiments

Long-term assimilation experiments were performed during two periods: boreal winter (1 January–10 February 2004 ) and autumn (1 September–10 October 2003), comprising 41 and 40 days respectively. In total, eight experiments are discussed here, corresponding to the four cases

  • SB,

  • CB,

  • SBQG and

  • CBQG.

Assimilation cycles involving scheme LB were also executed; however, since the results were qualitatively similar to case CB (and because we wish to focus on flow-dependent aspects of the analysis evolution) they will not be presented. Each cycle included its own background check and in all cases the background-error variances and spatial correlations used were the same as those in the control case SB. If these statistics were re-derived for each particular balance, a larger impact on the final increments might be expected; however, as shown in section 2.2, the balanced temperature standard deviation calculated with operators CB and SB is quite similar. Therefore we expect that this adjustment would not have a significant effect on the assimilation. For brevity, we limit the discussion to those results that show the largest differences and to those comparisons that are most relevant to the theme of the article.

Various diagnostics were applied globally and also for the latitude bands 30°N–90°N (Northern Hemisphere), 30°S–30°N (Tropics) and 90°S–30°S (Southern Hemisphere). One metric that we examined was the overall amplitude of analysis increments. In Figure 7 we plot the difference of the temporal root-mean-square (RMS) of increments for variables T and u at 500 hPa and for ps at the surface (η = 1), computed over the winter period. The difference shown is cycle CB minus cycle SB, so that negative values (dark shades) correspond to regions where the CB constraint result in smaller increments, on average, than constraint SB. Although the differences are fairly small, overall the plots show a greater proportion of negative regions than positive.

thumbnail image

Figure 7. Difference in temporal RMS of increments for (a) δT [K] at 500 hPa, (b) δu [m s−1] at 500 hPa and (c) δps [Pa] over the period January 2004. The difference is cycle CB minus control cycle SB, i.e. the effect shown is due to the introduction of the Charney constraint. Dark (light) shades correspond to regions in which the RMS of the experiment is smaller (larger) than that of the control.

Download figure to PowerPoint

The vertical distribution of increment amplitudes during the entire assimilation period may be surmised from Figure 8, which shows the mean and standard deviation of increments δZ and δu calculated over the globe and the Tropics, as a function of pressure. The summation in this case is over both time and space. Constraint CB has a neutral effect on means over the globe and on the δu mean in the Tropics. In fact, the increments seem to be largely unaffected in the troposphere. However, a significant decrease is evident in the δZ mean in the Tropics above approximately 100 hPa. Likewise, the CB constraint reduces the standard deviations for both variables in the stratosphere, in the Tropics as well as globally. The fact that standard deviations are reduced more than bias suggests that the flow-dependent constraint leads to an improved representation of transient events. We find that such reductions generally increase with altitude. Results for the Northern and Southern Hemispheres (not shown) are similar to those in Figure 8.

thumbnail image

Figure 8. Mean (dashed curves) and standard deviation (continuous curves) over (a) and (b) the globe and (c) and (d) the Tropics of the increments (a), (c) δZ [m] and (b), (d) δu [m s−1] during January 2004 corresponding to (black curves) control cycle SB and (grey curves) cycle CB, i.e. the effect shown is due to the introduction of the Charney constraint.

Download figure to PowerPoint

In order to assess the accuracy of the new balance constraints more directly, the above diagnostics were also applied to the unbalanced components of the increments. In the Northern Hemisphere, constraint CB tends to produce somewhat larger unbalanced temperature increments than the control, however in the Tropics and Southern Hemisphere it reduces the standard deviations over much of the vertical extent of the domain, while the means remain roughly the same. As for the unbalanced surface pressure, δps,u, we plot the differences of the temporal RMS fields in Figure 9 for the winter period. Both plots are dominated by dark-shaded regions, which suggests that, on average, smaller δps,u increments are induced by the modified mass–wind and vorticity–divergence balance. It is interesting to note that the pattern of RMS differences is quite similar in both seasons (autumn not shown).

thumbnail image

Figure 9. Difference in temporal RMS of the unbalanced pressure increment δps,u [Pa] during January 2004. The difference is (a) cycle CB minus control cycle SB and (b) cycle SBQG minus control cycle SB, i.e. the effect shown is due to the introduction of (a) the Charney constraint and (b) the QG omega constraint. Dark (light) shades correspond to regions where the RMS of the experiment is smaller (larger) than that of the control.

Download figure to PowerPoint

3.3. Verifications against radiosondes

Scores against radiosonde observations were computed for analyses (O–A) and 6 h forecasts (O–F6) based on the assimilation cycles performed. Because the balance constraints affect the weighting of background versus observations, it is useful to examine both types of scores. Improvements in the fit to radiosonde measurements, for example, cannot be deemed meaningful if the quality of trial fields deteriorates concurrently, and vice versa. Only co-located radiosonde observations were used in these calculations. Global O–A and O–F6 statistics (not shown) indicate some improvement in case CBQG compared with case SB, particularly in the wind field in the upper troposphere/lower stratosphere; however, in general scores are very similar regardless of the constraint used.

On the other hand, in the Tropics the differences are more pronounced and persistent, as shown in Figure 10 for T and u during the winter period. Standard deviations for both variables improve in the lower stratosphere, comparing case CBQG and the control. Mean scores for the zonal velocity also improve quite significantly above approximately 700 hPa, and results are similar for the meridional component (not shown). The improvement in scores is significant at a number of pressure levels, as demonstrated by the confidence intervals (greater than 90%) given in shaded boxes at the sides of each plot. The left (right) confidence interval shown, as well as the shading of its box, corresponds to the experiment with the lower bias (standard deviation). Almost all of the boxes are grey, which provides a quick visual check that experiment CBQG leads to lower scores than the control, SB. During the autumn period, the beneficial impact of the CBQG constraint on u global-mean scores is as large as during winter, although the impact on other statistics is more neutral. Comparison of cases SB and SBQG as well as CB and CBQG reveals that most of the impact on the mass and wind fields seen in Figure 10 is due to the CB constraint, not the QG constraint, as expected.

thumbnail image

Figure 10. (a) and (b) O–A and (c) and (d) O–F6 scores against radiosondes as a function of pressure [hPa] computed over the Tropics for (a), (c) temperature [K] and (b), (d) zonal velocity [m s−1] based on cycles (black curves) SB and (grey curves) CBQG, during the winter period. The bias (dashed curves) and standard deviation (continuous curves) are shown, with confidence levels above 90% indicated in shaded boxes along the left (right) border of each plot.

Download figure to PowerPoint

To see if the impacts seen in 6 h forecasts extend to longer ranges, five-day forecasts were launched from the analyses produced by each cycle every 12 h for 35 days. Since each ensemble of forecasts has only 70 members, comparisons at longer lead times are unlikely to be meaningful. Scores against radiosondes were computed at 24, 48 and 120 h over various geographic regions. Since differences between ensembles tend to increase more-or-less monotonically with lead time, here we present results at 120 h where differences are greatest.

Similarly to O–A and O–F6 scores, discussed above, the largest contribution is due to balance constraint CB and we therefore focus on this case. Scores for the globe and the Tropics, comparing cases SB and CB during the winter period, are shown in Figures 11 and 12, respectively. The effect on global statistics is a small but statistically significant improvement in zonal velocity, wind magnitude and geopotential standard deviations between the mid-troposphere and lower stratosphere. A benefit is also evident in the zonal wind bias in the lower stratosphere and in temperature statistics at a few scattered levels throughout the atmosphere. The overall change is modest; however, even where the curves are very close together the shading of the confidence intervals suggests a small improvement due to the CB constraint (grey) over constraint SB (black).

thumbnail image

Figure 11. Global forecast verification scores against radiosondes at 120 h as a function of pressure [hPa] based on cycles (black curves) SB and (grey curves) CB during the winter period for (a) zonal velocity [m s−1], (b) wind modulus [m s−1], (c) geopotential height [dam] and (d) temperature [K]. Dashed (solid) curves correspond to bias (standard deviation), with confidence levels above 90% indicated in shaded boxes along the left (right) border of each plot.

Download figure to PowerPoint

thumbnail image

Figure 12. As Figure 11 but in the Tropics.

Download figure to PowerPoint

As with trial fields, the greatest impact of the new constraints on multiday forecasts is in the Tropics. Figure 12 demonstrates significant improvement due to the CB constraint in the wind and temperature standard deviations in the lower stratosphere. A dramatic decrease in the zonal velocity bias is evident throughout the atmosphere and a smaller but still prominent decrease in temperature bias in the stratosphere. As in previous figures, we note the preponderance of grey confidence interval boxes compared with black boxes, confirming that the experiment generally exhibits the lower scores, even at pressure levels where the curves seem to be coincident. While not shown, the effect on scores in the Northern (Southern) Hemisphere tends to be neutral (somewhat negative), so that the Southern Hemisphere mostly offsets the gains in the Tropics, resulting in the largely neutral scores shown in Figure 11.

The response in the Southern Hemisphere may be a symptom of the flawed vertical structure of balanced temperature in high latitudes in the CB and CBQG implementations, as suggested in section 2.3. Presumably this difficulty is not as detrimental in the Northern Hemisphere, because analyses in this region are more strongly constrained due to the greater availability of observations. Finally, forecast verifications in the autumn period (not shown) exhibit a pattern that is similar qualitatively to the winter period, but smaller in amplitude.

In the stratosphere, particularly in the Tropics, the decrease in increment amplitudes (Figure 8) and objective scores (Figures 1012) associated with constraint CB may be due to a somewhat better representation of variability resulting from the Charney balance, as suggested by Figure 1. Although the δTb variance in Figure 1(c) is significantly different from the true δT variance (Figure 1(a)) in the stratosphere and lower troposphere, it nevertheless seems closer than the control (Figure 1(b)). In particular, the arch-shape along the tropopause in (a) is better resolved in (c) than in (b). Since the tropopause is a source of gravity waves (which heavily influence stratospheric dynamics), capturing the correct variability in the upper troposphere is likely to be beneficial for the stratosphere. Also, (c) exhibits somewhat lower (higher) variance in the Southern Hemisphere (Tropics) than (b) between 100 and 1 hPa, in qualitative agreement with (a).

3.4. Verifications against analysis

The synergy between the model, assimilation scheme and observations may be further evaluated by comparing forecasts with their respective analyses. While this is not an objective characterization of the forecast skill, it does provide another way to compare modifications to the system, within the confines of imperfect components with their respective biases, both known and unknown. The difference between the forecast and analysis valid at the same time was computed, and this will be called the forecast error. As above, we focus our attention on the longest lead time available (5 days).

In Figure 13 we show the mean and standard deviation of the T and u forecast error for control case SB and experiment CBQG at a lead time of 120 h in the Northern Hemisphere and Tropics during the winter period. In the troposphere, the new balance constraint has a negligible impact; however, quite noticeable changes can be seen in the stratosphere, with a positive impact on the variance of both fields. The T and u standard deviation profiles are improved in both regions above approximately 10 and 50 hPa, respectively, along with improved T bias in the Northern Hemisphere at upper levels. The effect in the Southern Hemisphere (not shown) is smaller, but nevertheless positive. As with previous diagnostics, the amplitude of the effect in the autumn period was found to be smaller than in the winter period, and its sign may reverse depending on the level. This suggests that the system will respond differently to the flow-dependent constraints as the structure of the atmosphere (particularly the stratosphere) changes with the seasons.

thumbnail image

Figure 13. Mean (dashed curves) and standard deviation (continuous curves) of the forecast–analysis error for (a) and (b) temperature T [K] and (c) and (d) zonal velocity u [m s−1] at a lead time of 120 h over (a), (c) the Northern Hemisphere and (b), (d) the Tropics during the winter period for control cycle SB (black curves) and cycle CBQG (grey curves).

Download figure to PowerPoint

3.5. Computational cost

Table 1 gives estimates of the average computational cost associated with constraints SB, CB and CBQG per analysis, during both winter and autumn periods, in terms of iterations and simulations. A simulation is a single attempt by the minimization algorithm to decrease the value of the cost function. An iteration, consisting of one or more simulations, is deemed to be complete when the value of the cost function has been successfully decreased. For a well-written algorithm, the number of simulations Nsim is expected to be only slightly higher than the number of iterations Niter, as is true for all three cases presented here. The number of iterations is significantly smaller for the CB and CBQG cases than for the control case SB, which suggests that the flow-dependent covariances result in faster convergence. We also examined the final value of the 3D-Var cost function in all experiments. Although the differences are small, on average, the cost function decreases when constraint SB is replaced by CB and decreases further in case CBQG.

However, the 3D-Var execution time, also shown in Table 1, is approximately three times higher with constraint CB than with constraint SB. Not surprisingly, the solution of several non-trivial equations takes much longer than the application of constant, pre-computed covariances. With the addition of the QG omega balance, the number of iterations remains the same, however the final execution time jumps by another factor of two. In an operational setting, such an increase in computational expense could only be justified if the new balance operator provided dramatic improvements in forecasting skill. As we have seen, the current adiabatic formulation does not offer such a benefit; however, more substantial gains may be possible if a diabatic heating is included (Fillion et al., 2005). It should be stressed that the numerical procedures used in this study have not been optimized in any way. The focus of this work has been to solve the balance relations as rigorously as possible and to assess their impact on analyses and forecasts. The efficiency of the code is an issue that could be addressed in the future.

Table 1. Average number of iterations Niter, average number of simulations Nsim and average duration T (minutes) of 3D-Var for a 6 h assimilation window, based on cycles SB, CB and CBQG during the September 2003 (autumn) and January 2004 (winter) periods.
 AutumnWinter
 SBCBCBQGSBCBCBQG
Niter645556786565
Nsim706161857171
T (min)82555112964

4. Discussion and conclusions

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methodology and validation
  5. 3. Experiments with 3D-Var
  6. 4. Discussion and conclusions
  7. Acknowledgements
  8. Appendix: Tangent Linear Equations
  9. References

The linearized Charney balance and QG omega equation have been introduced as balance constraints into a 3D-Var assimilation scheme similar to the operational forecast system at EC. Idealized, single-observation assimilation experiments demonstrate that these new covariance models lead to flow-dependent increments, as shown previously by Fisher (2003). Five-week assimilation cycles using standard meteorological observations demonstrate that the impact of the new constraints on forecast scores against radiosondes is qualitatively neutral in most regions of the globe. The exceptions are the Southern Hemisphere, where the response is mostly negative, especially for the temperature, and the Tropics, where the response is decidedly positive for all analysis variables in terms of O–A and O–F statistics at all lead times. Forecast verifications against analyses show only a weak impact in the troposphere but a robust reduction of forecast errors in the stratosphere.

The improvements observed are mainly due to the linearized Charney balance, whereas the role of the QG omega constraint tends to be marginal, at least for the parameter values we have chosen. In particular, the static stability may have been too large for this application, which suppressed the balanced component of vertical motion. Compared with control cases, the new balance operators require a significant amount of computational resources. The QG omega equation in particular increases the duration of a single 3D-Var assimilation step by a factor of 2–3. We note, however, that the solution techniques have not been optimized and the QG omega equation may prove more useful if it is forced diabatically (Fillion et al., 2005).

This research has revealed several unexpected aspects of the methodology introduced in Fisher (2003). Unlike the static covariances, the Charney and QG omega operators do not involve averaging in time or space and thus tend to produce relatively noisy fields. The resulting small-scale features may themselves spawn spurious gravity waves, which could partially explain the lack of substantial improvement in forecast skill. Again it should be noted that in the assimilation experiments described here only cross-covariances were modified, whereas variances and spatial correlations were the same in all cases (and based on regression). Although it would be valuable to perform experiments in which all the error statistics were self-consistent, the impact on forecasts is unlikely to be dramatic since, as we have seen (Figure 1), temperature standard deviations based on forecast ensembles were similar for both the SB and CB cases.

One of the chief difficulties with the approach presented here has turned out to be a fundamental property of mass–wind balance relations, namely that neither the mass variable nor the wind variable can be inverted reliably to provide balanced fields at all spatial scales and in all regions of the globe. In our investigation, we have retained the formulation of the EC GDPS, in which the TL equations associated with the dynamical constraints compute balanced fields from the stream-function increment. This framework can be expected to work well on mesoscales/synoptic scales and in the Tropics (Lorenc et al., 2003; Wlasak et al., 2006). Indeed, our results are consistent with this expectation in that the new constraints provide a robust improvement in objective forecast skill, primarily in the Tropics. Moreover, the same methodology appears to be beneficial in the case of regional data assimilation, which involves relatively small spatial scales (e.g. Barker et al., 2004).

Outside the Tropics, on scales much larger than the Rossby deformation radius, the unbalanced component of stream-function increments is quite significant and projects strongly on to the other control variables (Wlasak et al., 2006). A high-pass spectral filter with a fixed wave-number cut-off, as applied in this study, eliminates most of the planetary-scale unbalanced signal; however, the resulting balanced temperatures are still found to be contaminated to some degree. This undoubtedly affects increments adversely and is likely to mask some of the beneficial influence of flow dependence. Introducing a Burger-number-dependent spectral filter could potentially alleviate this difficulty, at the cost of increased algorithm complexity and computer time.

The Fisher (2003) constraint, formulated in terms of wavelets, was investigated at ECMWF and eventually became operational (Fisher, 2004; IFS, 2006). Certain mathematical approximations that we adopted in the present study in order to make the solution scheme tractable are liable to induce biases of their own. However, since to the best of our knowledge similar simplifications were made in the ECMWF operational implementation, this may not be enough to explain the largely neutral impact on objective scores that we have observed. It is possible that a wavelet-based formulation is essential, since it allows for spatial as well as spectral dependence (see, for example, the idealized experiments in Bannister, 2007). This would suggest that the details of implementation are quite important.

The potential vorticity (PV) implicitly accounts for the scale dependence of the mass–wind balance, and it is chiefly for this reason that PV-based constraints have received much attention (Cullen, 2003; Wlasak et al., 2006; Katz et al., 2011). It seems, however, that their implementation is far from trivial and suffers from at least some of the obstacles described in this study. Cullen (2003), for example, found that increments developed significant oscillatory anomalies in the vertical structure, which was believed to be linked to the particular vertical staggering scheme used by the forecast model. Although the PV approach has the desirable property that it allows for an unbalanced component of the stream function, it is still not obvious how to prevent this mode from contaminating the mass field. A technique based on spectral filtering has been discussed in Bannister (2009). We suspect that the application of balance relations to increments, rather than analysis fields, is a further complication since, as we have shown, temporal differences of dynamical fields tend to be significantly more unbalanced than the fields themselves.

Flow dependence may be introduced through a normal mode decomposition, treating rotational and irrotational modes as balanced and unbalanced, respectively (Žagar et al., 2004). Kleist et al. (2009) have recently reported fairly promising results using this method in a 3D-Var system. Even though our investigation was performed in the context of 3D-Var, the issues are relevant for 4D-Var systems. In a typical 4D-Var scheme the covariances are evolved forward in time but only during an assimilation window (i.e. 6 h), whereas at the start of each assimilation window they are typically reset to static, regression-based covariances (Bannister, 2008b). Therefore, the impact of balance relations will be smaller than in the case of 3D-Var; however, flow dependence in the statistics may still have a beneficial, non-negligible effect.

Ensemble-based methods (Buehner, 2005; Buehner et al., 2010a, 2010b) allow flow dependence without explicitly solving any dynamical balance relation. However, they rely on the concurrent generation and storage of large ensembles of forecasts. Indeed, in an operational setting, even producing static, NMC-based statistics typically requires an ensemble for each month of the year. Modelled covariances, on the other hand, have no such requirements and are an attractive option for that reason. However, as discussed in this article, constraints that appear to be advantageous on theoretical grounds do not necessarily lead to marked improvements in forecast skill and their success or failure depends heavily on the particular implementation chosen.

Acknowledgements

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methodology and validation
  5. 3. Experiments with 3D-Var
  6. 4. Discussion and conclusions
  7. Acknowledgements
  8. Appendix: Tangent Linear Equations
  9. References

The authors express their sincere thanks to their Environment Canada colleagues Luc Fillion (ARMA), Yves Rochon (Air Quality Research Division, Experimental Studies Section, ARQX), Mark Buehner (ARMA) and Cécilien Charette (ARMA) for their help with the many theoretical and technical aspects of this research. Suggestions made by two anonymous reviewers on ways to improve the manuscript are also greatly appreciated. Much of the analysis and plotting during the course of this project was done using the GrADS software package.

Appendix: Tangent Linear Equations

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methodology and validation
  5. 3. Experiments with 3D-Var
  6. 4. Discussion and conclusions
  7. Acknowledgements
  8. Appendix: Tangent Linear Equations
  9. References

Let the temperature, geopotential, horizontal velocity, vorticity, pressure, surface pressure, omega field, stream function and velocity potential be given by T, Φ, u = (u, v), ζ = ∇2ψ, p, ps, ω, ψ and χ, respectively, in a coordinate system with zonal, meridional and vertical components (λ, ϕ, η). The reference state will be indicated by subscript ‘ref’ and balanced fields by subscript ‘b’, while increment fields will be preceded by δ. An overbar indicates a global mean. Then the linear model associated with the balance constraint implemented in our experimental 3D-Var scheme can be written as

  • equation image(A1)
  • equation image(A2)
  • equation image(A3)
  • equation image(A4)
  • equation image(A5)

where f = 2Ωsinϕ is the Coriolis parameter, β = 2Ωcos(ϕ)/a, a is the Earth's radius, Ω is the Earth's angular speed of rotation and R is the dry gas constant. The static stability parameter is defined as

  • equation image(A6)

where equation image and equation image are reference profiles for the temperature and potential temperature, while

  • equation image(A7)

and

  • equation image(A8)

follow from the definition of the model vertical coordinate η, in which p0 = 800 hPa is a constant reference pressure, ηT = pT/p0, pT = 0.1 hPa is the pressure at the top of the model atmosphere and r = 1.6 (Charron et al., 2011).

Equations (A1)–(A4) are the linearized Charney, hydrostatic, QG omega and continuity equations, respectively, corresponding to equations (11.15), (6.2), (6.29) and (6.3) in Holton (1992), after transformation from pressure to the hybrid coordinate η and after a number of simplifying assumptions discussed below. Equation (A5) for the balanced pressure increment at the surface (η = 1) was derived by adapting the procedure in Mitchell et al. (2002) for Charney's balance. Neglecting the uref-dependent terms on the right-hand sides of (A1) and (A5) yields the so-called linear balance. The background vorticity is given by

  • equation image(A9)

and it is henceforth assumed that uref is the rotational wind, i.e.

  • equation image(A10)

with analogous expressions for the linear perturbations δζ and δu in terms of δψ:

  • equation image(A11)

Transforming to spherical harmonics, inverting the elliptic operators on the left-hand sides of (A1), (A3) and (A4) and using (A11), we may rewrite (A1)–(A4) (combining (A5) with (A2)), respectively, as

  • equation image(A12)
  • equation image(A13)
  • equation image(A14)
  • equation image(A15)

which implicitly defines operators C, V1, P, Q1, Q2 and V2. Thus we have N2 = (V1,P), E2 = V2Q1 and E3 = V2Q2.

A number of approximations and simplifications were made in the derivation. Many of these were investigated numerically using model 6 h differences as outlined in section 2.3 and were found to have a negligible impact on the solution, at least within the level of error considered here. Moreover, a completely rigorous application of the balance equations is neither practical nor feasible, given the necessity to implement and maintain a corresponding adjoint model as well as the significant computational resources required to solve the equations. To the authors' knowledge, similar assumptions were made in the study by Fisher (2003). In our idealized, offline tests (section 3.) the quantity δT in (A3) is approximated by δTB, since an unbalanced component δTU is not available.

As is usually done, we have neglected metric corrections due to spherical geometry in (A1) (Houghton, 1968), which were found to be relatively small. Moreover, rigorous transformation of the balance equations from pressure to model coordinates requires correction terms such that, for zonal derivatives,

  • equation image(A16)

and similarly for meridional derivatives (Kasahara, 1974). The only such corrections we have retained are the four terms on the right-hand side of (A3) that involve ps. These result from transformation of the advection terms and were found to make a non-negligible contribution to δωb. Vertical derivatives are subject to the simplified transformation

  • equation image(A17)

with α given by (A7). The parameter σ was taken as a constant in all experiments, with a nominal tropospheric value of 1 × 10−6 m4 s2 kg−2 as in, for example, Bluestein (1992) and Tsou et al. (1987). Although a vertically varying stability parameter would be more realistic and could potentially improve the solution to (A3), it would require a significantly more involved numerical procedure.

Equation (A5) was obtained by allowing for variations of the surface pressure in (A1) and solving for δps,b. However, this dependence was not retained in the final formulation of (A1)–(A4). Simple trials with model 6 h differences revealed that the additional corrections associated with δps,b have little impact on δTb in the troposphere, while in the stratosphere they cause large, unphysical anomalies. It is likely that the numerical error incurred in computing these terms grows with height and dominates the true signal at higher altitudes, which is one of the difficulties we encountered with this approach to covariance modelling.

Technically, the QG omega equation is only valid in a limited area where f is non-zero and deviates only slightly from some constant reference value. However, we apply (A3) globally, allowing the full variation of f. Although solutions were found to be generally well-behaved in both nonlinear and linearized regimes, it was found beneficial to taper the right-hand side of (A3) to zero near the Equator using a smooth damping function. (A rigorous theory that corresponds to global QG dynamics was derived by Schubert et al. (2009) and could be the basis of a future investigation in this context.) We add that Fisher (2003) adopted the Q-vector form of the QG omega equation (Hoskins et al., 1978), which exploits certain cancellations inherent in (A3). However, in our numerical tests the Q-vector form yielded rather noisy solutions (perhaps due to our particular method of computing spatial gradients, described in the Appendix) and we opted for the traditional form instead.

References

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methodology and validation
  5. 3. Experiments with 3D-Var
  6. 4. Discussion and conclusions
  7. Acknowledgements
  8. Appendix: Tangent Linear Equations
  9. References
  • Alexander MJ, Geller M, McLandress C, Polavarapu S, Preusse P, Sassi F, Sato K, Eckermann S, Ern M, Hertzog A, Kawatani Y, Pulido M, Shaw TA, Sigmond M, Vincent R, Watanabe S. 2010. Recent developments in gravity-wave effects in climate models and the global distribution of gravity-wave momentum flux from observations and models. Q. J. R. Meteorol. Soc. 136: 11031124. DOI:10.1002/qj.637.
  • Anderson E, Bai Z, Bischof C, Blackford S, Demmel J, Dongarra J, Du Croz J, Greenbaum A, Hammarling S, McKenney A, Sorensen D. 1999. LAPACK Users' Guide, 3rd edn; 407pp. Society for Industrial and Applied Mathematics: Philadelphia, PA.
  • Bannister RN. 2007. Can wavelets improve the representation of forecast error covariances in variational data assimilation? Mon. Weather Rev. 135: 387408.
  • Bannister RN. 2008a. A review of forecast error covariance statistics in atmospheric variational data assimilation. I: Characteristics and measurements of forecast error covariances. Q. J. R. Meteorol. Soc. 134: 19511970.
  • Bannister RN. 2008b. A review of forecast error covariance statistics in atmospheric variational data assimilation. II: Modelling the forecast error covariance statistics. Q. J. R. Meteorol. Soc. 134: 19711996.
  • Bannister RN. 2009. ‘What is the cut-off total wavenumber for each vertical mode of unbalanced pressure?’ Project Document. University of Reading: Reading, UK. http://www.met.reading.ac.uk/∼ross/DARC/PVcv/VertModes/CutOff.pdf.
  • Barker DM, Huang W, Guo YR, Bourgeois AJ, Xiao QN. 2004. A three-dimensional variational data-assimilation system for MM5: Implementation and initial results. Mon. Weather Rev. 132: 897914.
  • Beagley SR, de Grandpré J, Koshyk JN, McFarlane NA, Shepherd TG. 1997. Radiative–dynamical climatology of the First-Generation Canadian Middle Atmosphere Model. Atmos.-Ocean 35: 293331.
  • Bluestein HB. 1992. Synoptic–Dynamic Meteorology in Midlatitudes. Oxford University Press: Oxford; 431pp.
  • Bonavita M, Raynaud L, Isaksen L. 2011. Estimating background-error variances with the ECMWF Ensemble of Data Assimilations system: some effects of ensemble size and day-to-day variability. Q. J. R. Meteorol. Soc. 137: 423434.
  • Bregman B, Meijer E, Scheele R. 2006. Key aspects of stratospheric tracer modeling using assimilated winds. Atmos. Chem. Phys. 6: 45294543.
  • Buehner M. 2005. Ensemble-derived stationary and flow-dependent background-error covariances: Evaluation in a quasi-operational NWP setting. Q. J. R. Meteorol. Soc. 131: 10131043.
  • Buehner M, Houtekamer PL, Charette C, Mitchell HL, He B. 2010a. Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part I: Description and single-observation experiments. Mon. Weather Rev. 138: 18851901.
  • Buehner M, Houtekamer PL, Charette C, Mitchell HL, He B. 2010b. Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part II: One-month experiments with real observations. Mon. Weather Rev. 138: 19021921.
  • Caron J-F, Yau MK, Laroche S, Zwack P. 2007a. The characteristics of key analysis errors. Part I: Dynamical balance and comparison with observations. Mon. Weather Rev. 135: 249266.
  • Caron J-F, Yau MK, Laroche S, Zwack P. 2007b. The characteristics of key analysis errors. Part II: The importance of the PV corrections and the impact of balance. Mon. Weather Rev. 135: 267280.
  • Charney JG. 1963. A note on large-scale motions in the Tropics. J. Atmos. Sci. 20: 607609.
  • Charron M, Polavarapu S, Buehner M, Vaillancourt PA, Charette C, Roch M, Morneau J, Garand L, Aparicio JM, MacPherson S, Pellerin S, St-James J, Heilliette S. 2011. The stratospheric extension of the Canadian global deterministic medium range weather forecasting system and its impact on tropospheric forecasts. Mon. Wea. Rev. (in press).
  • Courtier P, Thépaut J-N, Hollingsworth A. 1994. A strategy for operational implementation of 4D-Var, using an incremental approach. Q. J. R. Meteorol. Soc. 120: 13671387.
  • Courtier P, Andersson E, Heckley W, Pailleux J, Vasiljevic D, Hamrud M, Hollingsworth A, Rabier F, Fisher M. 1998. The ECMWF implementation of three dimensional variational assimilation (3D-Var). I: Formulation. Q. J. R. Meteorol. Soc. 124: 17831807.
  • Cullen MJP. 2003. Four dimensional variational data assimilation: a new formulation of the background-error covariance matrix based on a potential vorticity representation. Q. J. R. Meteorol. Soc. 129: 27772796.
  • Dee DP. 2005. Bias and data assimilation. Q. J. R. Meteorol. Soc. 131: 33233343.
  • Dee SP, Uppala S. 2009. Variational bias correction of satellite radiance data in the ERA-Interim reanalysis. Q. J. R. Meteorol. Soc. 135: 18301841.
  • de Grandpré J, Ménard R, Rochon YJ, Charette C, Chabrillat S, Robichaud A. 2009. Radiative impact of ozone on temperature predictability in a coupled chemistry–dynamics data assimilation system. Mon. Weather Rev. 137: 679692.
  • Derber J, Bouttier F. 1999. A reformulation of the background error covariance in the ECMWF global data-assimilation system. Tellus. 51A: 195221.
  • Fillion L, Tanguay M, Ek N, Pagé C, Pellerin S. 2005. ‘Balanced coupling between vertical motion and diabatic heating for variational data assimilation’. In Proceedings of WWRP Symposium on Nowcasting and Very Short Range Forecasting, Toulouse, France, 5–9 Sep 2005. Meteo France: Toulouse, France. http://www.meteo.fr/cic/wsn05/.
  • Fillion L, Tanguay M, Lapalme E, Denis B, Liu Z, Ek N, Desgagne M, Lee V, Pellerin S. 2007. 'Case dependent implicit normal mode balance operators'. In Proceedings of ECMWF Workshop on Flow-dependent Aspects of Data Assimilation, Reading, United Kingdom, 11–13 Jun 2007. European Centre for Medium-Range Weather Forecasts: Reading, UK. http://www.ecmwf.int/publications.
  • Fisher M. 2003. ‘Background error covariance modelling’. In Proceedings of ECMWF Seminar on Recent Developments in Data Assimilation for Atmosphere and Ocean, Reading, United Kingdom, 18–12 Sep 2003. European Centre for Medium-Range Weather Forecasts: Reading, UK. http://www.ecmwf.int/publications.
  • Fisher M. 2004. ‘Generalized frames on the sphere, with application to background error covariance modelling’. In Proceedings of ECMWF Seminar on Recent Developments in Numerical Methods for Atmosphere and Ocean Modelling, Reading, United Kingdom, 6–10 Sep 2004. European Centre for Medium-Range Weather Forecasts: Reading, UK. http://www.ecmwf.int/publications.
  • Gauthier P, Charette C, Fillion L, Koclas P, Laroche S. 1999. Implementation of a 3D variational data-assimilation system at the Canadian Meteorological Centre. Part I: The global analysis. Atmos.-Ocean 37: 103156.
  • Gauthier P, Tanguay M, Laroche S, Pellerin S. 2007. Extension of 3DVAR to 4DVAR: Implementation of 4DVAR at the Meteorological Service of Canada. Mon. Weather Rev. 135: 23392354.
  • Hall TM, Plumb RA. 1994. Age as a diagnostic of stratospheric transport. J. Geophys. Res. 99D1 10591070, DOI: 10.1029/93JD03192.
  • Hollingsworth A. 1995. 'A spurious mode in the ‘Lorenz’ arrangement of ϕ and T which does not exist in the 'Charney–Philips' arrangement', ECMWF Technical Memo No. 211. European Centre for Medium-Range Weather Forecasts: Reading, UK.
  • Holton JR. 1992. An Introduction to Dynamic Meteorology. Academic Press: San Diego; 511pp.
  • Hoskins BJ, Draghici I, Davies HC. 1978. A new look at the ω-equation. Q. J. R. Meteorol. Soc. 73: 370383.
  • Houghton DD. 1968. Derivation of the elliptic condition for the balance equation in spherical coordinates. J. Atmos. Sci. 25: 927928.
  • IFS. 2006. IFS DocumentationCy31r1 (Part II: Data Assimilation). European Centre for Medium-Range Weather Forecasts: Reading, UK. http://www.ecmwf.int/research/ifsdocs.
  • Jackson DR, Keil M, Devenish BJ. 2008. Use of Canadian Quick covariances in the Met Office data-assimilation system. Q. J. R. Meteorol. Soc. 134: 15671582.
  • Kasahara A. 1974. Various vertical coordinate systems used for numerical weather prediction. Mon. Weather Rev. 102: 509522.
  • Katz D, Lawless AS, Nichols NK, Cullen MJP, Bannister RN. 2011. Correlations of control variables in variational data assimilation. Q. J. R. Meteorol. Soc. 137: 620630.
  • Kleist DT, Parrish DF, Derber JC, Treadon R, Errico RM, Yang R. 2009. Improving incremental balance in the GSI 3DVAR analysis system. Mon. Weather Rev. 137: 10461060.
  • Laroche S, Gauthier P, Tanguay M, Pellerin S. 2007. Impact of the different components of 4DVAR on the Global Forecast System of the Meteorological Service of Canada. Mon. Weather Rev. 135: 23552364.
  • Lindskog M, Vignes O, Gustafsson N, Landelius T, Thorsteinsson S. 2007. ‘Background errors in HIRLAM variational data assimilation’. In Proceedings of ECMWF Workshop on Flow-dependent Aspects of Data Assimilation, Reading, United Kingdom, 11–13 Jun 2007. European Centre for Medium-Range Weather Forecasts: Reading, UK. http://www.ecmwf.int/publications.
  • Lindskog M, Dee D, Trémolet Y, Andersson E, Radnóti G, Fisher M. 2009. A weak-constraint four-dimensional variational analysis system in the stratosphere. Q. J. R. Meteorol. Soc. 135: 695706.
  • Lorenc A, Roulstone I, White A. 2003. ‘On the choice of control fields in Var’, Technical Report No. 419; 36pp. Met Office: Exeter, UK.
  • Ménard R, Gauthier P, de Grandpré J, Robichaud A, Rochon Y, Chabrillat S, Fonteyn D, von Clarmann T, Yang Y, Charron M, McConnell J, Kaminski J, Vaillancourt P, Charette C, Kallaur A. 2007. ‘Coupled chemical–dynamical data assimilation’, Technical Report, ESA/ESTEC Contract No. 18560/04/NL/FF; 486pp. European Space Agency: Nordwijk, the Netherlands.
  • Mitchell HL, Houtekamer PL, Pellerin G. 2002. Ensemble size, balance and model-error representation in an ensemble Kalman filter. Mon. Weather Rev. 130: 27912808.
  • Monge-Sanz BM, Chipperfield MP, Simmons AJ, Uppala SM. 2007. Mean age of air and transport in a CTM: Comparison of different ECMWF analyses. Geophys. Res. Lett. 34: L04801, doi: 10.1029/2006GL028515.
  • Pagé C, Fillion L, Zwack P. 2007. Diagnosing summertime mesoscale vertical motion: Implications for atmospheric data assimilation. Mon. Weather Rev. 135: 20762094.
  • Parrish DF, Derber JC. 1992. The National Meteorological Center's Spectral Statistical Interpolation analysis system. Mon. Weather Rev. 120: 17471763.
  • Polavarapu SM. 1995. Divergent wind analyses in the oceanic boundary layer. Tellus. 47A: 221239.
  • Polavarapu S, Ren S, Rochon Y, Sankey D, Ek N, Koshyk J, Tarasick D. 2005. Data assimilation with the Canadian Middle Atmosphere Model. Atmos.-Ocean 43: 77100.
  • Rabier F, McNally A, Andersson E, Courtier P, Undén P, Eyre J, Hollingsworth A, Bouttier F. 1998. The ECMWF implementation of three dimensional variational assimilation (3D-Var). II: Structure functions. Q. J. R. Meteorol. Soc. 124 18091829.
  • Raynaud L, Berre L, Desroziers G. 2011. An extended specification of flow-dependent background-error variances in the Météo-France global 4D-Var system. Q. J. R. Meteorol. Soc. 137 607619.
  • Schoeberl MR, Douglass AR, Zhu Z, Pawson S. 2003. A comparison of the lower stratospheric age spectra derived from a general circulation model and two data-assimilation systems. J. Geophys. Res. 108D3 4113, DOI: 10.1029/2002JD002652.
  • Schubert WH, Taft RK, Silvers LG. 2009. Shallow water quasi-geostrophic theory on the sphere. J. Adv. Model. Earth Syst. 1: 2.
  • Tremolet Y. 2007. Model-error estimation in 4D-Var. Q. J. R. Meteorol. Soc. 133: 12671280.
  • Tsou C-H, Smith PJ, Pauley PM. 1987. A comparison of adiabatic and diabatic forcing in an intense extratropical cyclone system. Mon. Weather Rev. 115: 763786.
  • Uppala SM, Kallberg PW, Simmons AJ, Andrae U, da Costa V, Bechtold P, Fiorino M, Gibson JK, Haseler J, Hernandez A, Kelly GA, Li X, Onogi K, Saarinen S, Sokka N, Allan RP, Andersson E, Arpe K, Balmaseda MA, Beljaars ACM, van de Berg L, Bidlot J, Bormann N, Caires S, Chevallier F, Dethof A, Dragosavac M, Fisher M, Fuentes M, Hagemann S, Holm E, Hoskins BJ, Isaksen L, Janssen PAEM, Jenne R, McNally AP, Mahfouf J-F, Morcrette J-J, Rayner NA, Saunders RW, Simon P, Sterl A, Trenberth KE, Untch A, Vasiljevic D, Viterbo P, Woollen J. 2005. The ERA-40 re-analysis. Q. J. R. Meteorol. Soc. 131: 29613012.
  • Wlasak M, Nichols NK, Roulstone I. 2006. Use of potential vorticity for incremental data assimilation. Q. J. R. Meteorol. Soc. 132: 28672886.
  • Žagar N, Gustafsson N, Kallen E. 2004. Dynamical response of equatorial waves in four-dimensional variational data assimilation. Tellus 56A: 2946.