Accounting for model error in the Météo-France ensemble data assimilation system

Authors


Abstract

Since July 2008, a variational ensemble data assimilation system has been used operationally at Météo-France to provide background error variances ‘of the day’ to the operational 4D-Var assimilation of the global Arpège model. The current ensemble is run in a perfect model framework and estimated variances are inflated ‘offline’ (i.e. after the ensemble has been completed) to account for model errors. The inflation coefficient is tuned according to a posteriori diagnostics relative to the minimum of the cost function. In this study, the ‘offline’ variance inflation is replaced by an ‘online’ multiplicative inflation of 6 h forecast perturbations after each step of 6 h model integration. This allows the inflation information to be accounted for in the production of background perturbations with realistic amplitudes for the perturbed analysis steps.

In the case of a perfect model approach, background error standard deviations are underestimated by a factor of approximately two. When using online inflation to avoid this kind of mismatch, background perturbations after 6 h of model integration are inflated by around 10%. Examination of error spectra and of standard deviation maps indicates that the increase of variance is somewhat larger for synoptic scales and in data-sparse regions with dynamically active systems such as in the extratropical part of the Southern Hemisphere. Moreover, the reduction of background perturbation amplitude during the analysis step is more pronounced, especially for large-scale variables such as temperature and surface pressure.

Parallel analysis and forecast experiments indicate that the covariance estimates provided by the inflated background perturbations have a neutral to positive impact on the forecast quality, in addition to being more consistent with innovation-based estimates. Copyright © 2011 Royal Meteorological Society

1. Introduction

In numerical weather prediction (NWP), the model initial conditions, also known as the analysis state, are produced as a combination of observations and a background state, usually provided by a short-term forecast from a previous model run. These two sources of information are appropriately weighted during the assimilation process, according to their respective error covariances represented in the so-called R and B matrices respectively.

Background errors are the result of two different errors: a predictability error, which corresponds to the growth of analysis error during the model integration, and a model error (Daley, 1992). The model error term is due to imperfections in the numerical model, arising, for instance, from approximations in the dynamics and in the physical parametrizations.

Monte Carlo methods, based either on the Kalman filter algorithm (the ensemble Kalman filter; Evensen, 2003) or on variational data assimilation (Belo Pereira and Berre, 2006; Berre et al., 2007), are ongoing alternative or complementary approaches to deterministic assimilation systems, and they can provide estimates of flow-dependent background error covariances.

However, ensemble data assimilation systems are sometimes implemented in a perfect model framework. Consequently, they provide an estimation of the predictability error term while the model error component is omitted. As neglecting model error usually results in underestimated background error variances, the representation of model error has become an important aspect in the development of ensemble methods.

First investigations to account for model errors in ensemble systems lead to various strategies. Possible options include in particular multi-model/multi-physics approaches (Houtekamer et al., 1996), stochastic physics (Buizza et al., 1999), additive and multiplicative inflations (Constantinescu et al., 2007) or the stochastic kinetic energy backscatter scheme (Shutts, 2005). These different model error representations have recently been evaluated and compared by Houtekamer etal. (2009), which indicate that the inflation approach has the largest contribution in their model error simulation.

At Météo-France, an ensemble variational assimilation has been developed for the global operational model Arpège (Berre et al., 2007) and has been successfully used in real time since July 2008 to provide flow-dependent background error variances to the operational 4D-Var assimilation (Raynaud et al., 2011). The ensemble is run in a perfect model framework and estimated variances are inflated ‘offline’ (i.e. after the ensemble has been completed), on the basis of a posteriori diagnostics (Desroziers and Ivanov, 2001), to represent model error contributions. Multiplicative inflation is a simple and widely used technique to deal with all error sources of unknown origins.

The current ensemble system is, however, not entirely consistent since the offline multiplicative inflation applied to the variances is not accounted for in the background perturbation update. An improved configuration of the ensemble has therefore been developed, in which the multiplicative inflation is now performed within the ensemble by enlarging the amplitude of forecast perturbations. The aim of this paper is to describe the inflation procedure and to examine the effect of simulating model error in the ensemble system, in terms of both description of background errors and their impact on forecast scores.

The paper is organized as follows. Section 2 presents the variational ensemble data assimilation system developed at our centre. The adaptive tuning procedure of the multiplicative inflation factor and the updated ensemble configuration are detailed in section 3. Section 4 provides diagnostic results from a first experimentation of the perturbation inflation. Forecast scores of analysis–forecast experiments over a 3-week period, using background error variances from the inflated perturbations, are examined in section 5. Finally, conclusions and perspectives are given in section 6.

2. Description of the ensemble system

2.1. Arpège 4D-Var

The operational model Arpège of Météo-France (Courtier et al., 1991) is a global spectral forecast model that uses a stretched horizontal resolution emphasizing the European area, with a finer resolution over France (Courtier and Geleyn, 1988). The assimilation scheme is based on a multi-incremental strong-constraint 4D-Var (Veersé and Thépaut, 1998) with two successive minimizations performed at spectral resolutions T107 and T224 respectively.

The modelling of the background error covariance matrix follows the multivariate formulation of Derber and Bouttier (1999). In this formulation, a set of independent analysis variables is considered (made up of vorticity, unbalanced divergence, unbalanced temperature and surface pressure, and specific humidity), and the associated background error covariance matrix is assumed to be block-diagonal, with no correlation between the parameters. Autocorrelation matrices are modelled with a diagonal assumption in spectral space (Courtier et al., 1998), that leads to homogeneous correlation functions in grid point space. Climatological correlation spectra are computed offline with an ensemble approach over a 1-month period (Belo Pereira and Berre, 2006). On the other hand, space- and time-varying variances are derived from a real-time ensemble variational assimilation (as detailed in section 2.2) for vorticity and unbalanced variables (Raynaud etal., 2011), while humidity variances are calculated with a simple empirical formula as a function of background temperature and relative humidity (Rabier et al., 1998). Multivariate coupling in the space of model variables is then achieved by applying balance relationships (accounting for geostrophic and hydrostatic balances in particular). Such modelled background error statistics are then partly flow dependent, due to the specification of error variances ‘of the day’ and to the use of linearized versions of nonlinear and omega balance equations (Fisher, 2003) that depend on the background state.

2.2. Arpège ensemble data assimilation

An ensemble data assimilation system has been built upon the existing 4D-Var system (Berre et al., 2007), based on explicit perturbations of observations (consistent with observation errors) while the background states are implicitly perturbed (due to the assimilation cycling of perturbed observations). The current ensemble is formed of six independent 4D-Var assimilations with one inner loop at spectral resolution T107, and is run in real time at a lower and unstretched horizontal resolution compared to the deterministic system (T399L70 vs. T798L70).

This ensemble has been used routinely since July 2008 to provide space- and time-varying estimates of background error variances to the operational 4D-Var assimilation, in replacement of quasi-static variances. The extraction of useful flow-dependent information about the correlations is currently under investigation following studies by Fisher and Andersson (2001) and Pannekoucke et al. (2007, 2008) on a wavelet diagonal approach. Moreover, the ensemble has been coupled to the Arpège ensemble prediction system since December 2009 to provide initial state perturbations.

Estimates of background error variances are derived from the ensemble at each assimilation cycle for each analysis variable:

equation image

where equation image denotes the perturbed background of member k and N is the ensemble size. An objective spatial filtering is then applied to the variances before they are specified in the operational 4D-Var assimilation (Raynaud et al., 2008, 2009; Berre and Desroziers, 2010). The use of these flow-dependent variances in the modelled B matrix is a first step to introducing information on the ‘errors of the day’ and appears to be beneficial to the forecast quality (Berre et al., 2007; Raynaud et al., 2011).

2.3. Model error contributions

The ensemble assimilation is currently run in a perfect model framework. To account for model error contributions, the estimated background error variances are tuned according to a posteriori diagnostics based on an evaluation of the 4D-Var cost function at the minimum (Talagrand, 1999; Desroziers and Ivanov, 2001). This corresponds to a multiplicative inflation factor applied ‘offline’ (i.e. after the ensemble has been completed) to the ensemble variances. This offline variance inflation procedure is detailed in section 3.1.

3. Perturbation inflation procedure

An alternative approach to the offline variance inflation is the use of an online inflation of forecast perturbations. The methodology of this perturbation inflation can be presented as a three-step approach.

The first step consists in estimating forecast error variances using observation-based diagnostics. In a second step the comparison with ensemble-estimated variances leads to the estimation of the inflation factor α. The third step corresponds to the multiplicative inflation of forecast perturbations by the factor α. These three steps are detailed in the following paragraphs.

3.1. Innovation-based estimation of forecast error variances

As discussed in Daley (1992), the forecast error ef can be written as the sum of predictability errors Mea (which correspond to analysis errors ea evolved by the forecast model M) and of model errors em:

equation image(1)

As shown by Desroziers and Ivanov (2001) and Chapnik et al. (2004), the forecast error variances can be estimated using innovation-based diagnostics, with a simple multiplicative operation:

equation image(2)

where Vspecified corresponds to background error variance values which are specified in the analysis before calculating the tuning coefficient sb.

The scalar tuning coefficient sb is calculated as the ratio between the observed minimum of the background cost function equation image and the theoretical minimum of the background cost function E[Jb(xa)] (E stands for the expectation operator):

equation image(3)

where E[Jb(xa)] is the expected statistical value that Jb would have if the system was optimal (i.e. if the B and R matrices were correctly specified). As shown by Talagrand (1999) and Desroziers and Ivanov (2001):

equation image(4)

where H is the linearized observation operator, K = BHT(HBHT + R−1) is the specified gain matrix and Tr denotes the trace operator. In practice, equation image is directly available from a 4D-Var deterministic run, and E[Jb(xa)] can be calculated directly from the ensemble data assimilation system (Desroziers et al., 2009).

The offline variance inflation mentioned in section 2.3 is restricted to this first step. While associated global values of background error variances are expected to be realistic, in the sense that they are consistent with innovation-based estimates, the associated ensemble system is not fully consistent. This corresponds to the fact that the model error information associated with innovation-based estimates is not integrated in the calculation of forecast perturbations. This implies in particular that analysis perturbations are calculated from background perturbations which are too small.

This limitation of the offline variance inflation approach can be avoided with an online inflation of forecast perturbations, which relies on two additional steps.

3.2. Calculation of the inflation factor

The ensemble assimilation system produces perturbed analyses whose spread is an estimate of analysis errors. As discussed in Daley (1992), the associated perturbed forecasts enable predictability error variances v[Mea] to be estimated.

Forecast error variances v[Mea + em] corresponding to Eq. (2) can be compared to predictability error variances v[Mea], by calculating the following global factor α for variables p such as temperature at a given time t:

equation image(5)

where equation image denotes a global spatial average over all horizontal grid points and vertical levels. Note that α is expected to be larger than 1.

Because α2 is a ratio between respective variances of forecast errors and of predictability errors, α can be seen as indirectly reflecting the contribution of model errors to forecast error variances. In particular, the larger the model error amplitude is, the larger α is.

3.3. Inflation of forecast perturbations

This factor α can be used to inflate perturbed 6 h forecasts as follows:

equation image(6)

This results in an increase of ensemble-based covariances by α2, while the ensemble mean is unchanged. This approach is known as the multiplicative inflation procedure (Anderson, 2001, 2007, 2009). In this study, the inflation factor was calculated for temperature, and this inflation factor was applied to inflate forecast perturbations of all variables (temperature, logarithm of surface pressure, vorticity and divergence) except humidity, which was not inflated in the experiments. This separate treatment of humidity is related to the fact that specified humidity error variances were calculated with an empirical formula, as discussed in section 2.1.

Compared to the offline variance inflation presented in section 2.3, this approach enables model perturbations to be represented and cycled in the ensemble.

While previous studies successfully applied other a posteriori diagnostics to adaptively tune the inflation coefficients, based for instance on innovation statistics (Miyoshi, 2005) and on observation-background/analysis-background statistics (Desroziers et al., 2005; Li et al., 2009), this is the first time, to the authors' knowledge, that diagnostics relative to the minimum of the cost function have been used to estimate a perturbation inflation factor in ensemble data assimilation systems.

It is worth noting that this multiplicative inflation is equivalent to assuming that the model error over the 6 h model integration is proportional to the growth of analysis error. This is an implication of Eq. (6), which can be seen as corresponding to the inflation of predictability perturbations by the factor α:

equation image(7)

where equation image are analysis perturbations evolved by the imperfect model M, and equation image are inflated forecast perturbations. Equivalently, Eq. (7) means that forecast perturbations can be written as follows, by analogy with Eq. (1):

equation image(8)

where equation image correspond to model perturbations calculated from β = α − 1.

3.4. Experimental set-up

To test the proposed approach, a variational ensemble with an online adaptive perturbation inflation, as described in the previous paragraph, has been run over the period between 17 January 0000 UTC and 09 February 1800 UTC 2010. The operational ensemble, which is run in a perfect model framework, is taken as a reference for comparison purposes. In parallel, deterministic analysis–forecast experiments using flow-dependent background error covariances derived from the inflated background perturbations have been run over the same period to examine the impact of these new statistics on forecast skill.

4. Diagnostic studies

In this section, analysis and background error statistics, computed from the baseline perfect model ensemble and from the ‘inflated’ ensemble, are compared in order to examine the impact of the adaptive perturbation inflation technique on error variances.

4.1. Inflation factors and global standard deviations

Figure 1 shows the time series of inflation factors calculated for 96 analysis time steps (from 17 January 0000 UTC to 09 February 1800 UTC 2010). After a short spin-up, the inflation becomes fairly stable, with a value for β around 10%. These values are in accordance with previous studies in EnKF contexts (e.g. Miyoshi and Yamane, 2007; Houtekamer et al., 2009).

Figure 1.

Time series of global adaptive inflation factor for the period between 17 January 0000 UTC and 09 February 1800 UTC 2010.

Vertical profiles of horizontally averaged background error standard deviations derived from ensembles with (dashed line) and without (solid line) model error simulation are displayed in Figure 2 for vorticity, divergence, temperature and specific humidity. As expected, standard deviations estimated from the ‘inflated’ ensemble are larger than those from the baseline perfect model ensemble at all vertical levels, while their variations with height are similar. Global standard deviations are increased by approximately a factor of 2 for vorticity, divergence and temperature. This indicates that the simulation of model error has a non-negligible impact on the amplitude of background errors in our system. It is worth noting that, while humidity background perturbations are not directly inflated, standard deviations are nevertheless increased by 25% on average, due to the inflation applied to the other variables.

Figure 2.

Vertical profiles of background error standard deviations derived from a perfect model ensemble (solid line) and from an ensemble with an online perturbation inflation (dashed line). The results are for (a) vorticity: s−1; (b) divergence: s−1; (c) temperature: K and (d) specific humidity: g kg−1. The left ordinate axis represents the model level, with level 70 being near the surface.

To check the realism of this global increase of ensemble background error standard deviations when the effects of model error are represented, they have been compared to independent estimates, derived from covariances of analysis residuals, as proposed by Desroziers et al. (2005):

equation image(9)

where equation image is the innovation vector and equation image is the analysis increment in observation space. The left-hand side term has been evaluated for different observation types and averaged over a 7-day period. Figure 3 shows that neglecting model error leads to an underdispersive ensemble, while simulating model error with a multiplicative inflation of background perturbations enables the ensemble spread to better fit the diagnosed errors. More precisely, this comparison indicates that the fit is fairly good in the mid and high troposphere (namely between 700 and 200 hPa) for the different data types shown, while the inflated standard deviations tend to underestimate (overestimate) the actual background error standard deviations in the lower troposphere (stratosphere).

Figure 3.

Vertical profiles of ensemble-based background error standard deviations in observation space, computed from a perfect model ensemble (dashed-dotted) and from an ensemble including an inflation of background perturbations (solid). These profiles are compared to diagnosed values from innovation-based statistics (Eq. (9), dashed). Results are given for temperature and wind observations from aircraft reports (AIREP) and radiosondes (TEMP and PILOT).

4.2. Horizontal variance spectra

In order to examine the contribution of the different horizontal scales to background errors and the extent to which they are affected by the model error simulation, horizontal variance spectra of background errors are displayed in Figures 4 and 5. Results are given for the logarithm of surface pressure and for vorticity and temperature near 500 hPa.

Figure 4.

Top panels: horizontal variance spectra of background error for (a) vorticity (s−2) and (b) temperature (K2) at 500 hPa. Bottom panels: corresponding normalized horizontal variance spectra. The results are for a perfect model ensemble (solid line) and for an ensemble with an online perturbation inflation (dashed line). Background errors are calculated at truncation T399.

Figure 5.

(a) Horizontal variance spectra of background error of the logarithm of surface pressure for a perfect model ensemble (solid line) and for an ensemble with an online perturbation inflation (dashed line). (b) Horizontal variance spectra of background error (dashed line) and analysis error (solid line) of the logarithm of surface pressure for an ensemble with an online perturbation inflation. Background and analysis errors are calculated at truncation T399. Unit: 1.

It may be mentioned that Figure 2 is related to Figures 4 and 5 through the relation equation image where n is the total wavenumber, l is a given model level, Vl(n) is the corresponding spectral variance and Nmax is the truncation wavenumber. The scales resolved by the global model Arpège can be grouped into three categories based on the wavenumber n: large or planetary scales associated with long waves for n between 1 and 5, synoptic scales for n between 6 and 20 and mesoscales for n > 20.

In the perfect model framework, one can first notice that synoptic scales have the largest contribution to temperature and surface pressure background errors (Figures 4(b) and 5(a)), while vorticity errors are dominated by the contribution from mesoscales (Figure 4(a)).

The simulation of model error contributes to increasing the amplitude of the whole spectrum accordingly. However, the normalized spectra (which represent the quantity equation image) in Figure 4(c) and (d) indicate that the enhancement of synoptic scales and mesoscales is more pronounced relative to planetary scales and small scales (wavenumbers greater than 100). The relative contribution decrease of planetary scales can be explained by the fact that the planetary scales are quite well resolved by the model dynamics on the one hand, and because they are efficiently corrected by the assimilation of new observations on the other. This is confirmed by the variance spectrum of analysis error of surface pressure, for instance (Figure 5(b)), which indicates a stronger ratio of error reduction in the planetary scales than in the smaller scales. The largest scale part of the background error is reduced by around 50% when model error is accounted for in the ensemble, while this reduction drops to 35% when model error is neglected. Regarding the smallest scales of the spectrum in Figure 4(c) and (d), the limited increase may be linked to saturation and dissipation by horizontal diffusion. Finally, synoptic scales of the spectrum are the most emphasized by the model error simulation since they correspond to dynamically active systems (such as baroclinic developments), which are more prone to rapid intensifications when inflated background perturbations are cycled in the nonlinear model. This predominant growth of errors in the synoptic scales is, moreover, consistent with previous results from Tribbia and Baumhefner (2004) and Hamill and Whitaker (2005), for instance.

4.3. Evolution of analysis and background errors

Figure 6 shows the temporal evolution of analysis and background errors for the logarithm of surface pressure and for vorticity, temperature and specific humidity at a level close to 500 hPa, for the period between 24 January 0000 UTC and 30 January 1800 UTC 2010. These error standard deviations have been computed from the baseline perfect model ensemble and from the new ensemble configuration, including a multiplicative perturbation inflation.

Figure 6.

Time series of analysis and background error standard deviations every 6 h for the period between 24 January 0000 UTC and 30 January 1800 UTC 2010. Statistics are derived from a perfect model ensemble (lower black line) and from an ensemble with an online perturbation inflation (upper black line). The crosses indicate standard deviations of analysis error, squares indicate background error standard deviations before inflation and circles indicate background error standard deviations after inflation. The results are for (a) vorticity at 500 hPa (s−1), (b) temperature at 500 hPa (K), (c) surface pressure (hPa), and (d) specific humidity at 500 hPa (g kg−1). This figure is available in colour online at wileyonlinelibrary.com/journal/qj

In a perfect model context, the statistics exhibit the typical pattern of an error growth with the dynamics of the model during the 6 h prediction step and a subsequent decrease due to the assimilation of observations. However, the error growth due to the model dynamics is relatively small compared to model error contributions represented in the ensemble with inflated perturbations, as can be seen in the upper black lines of Figure 6.

When model error is accounted for in the ensemble, the ensemble spread is increased by roughly a factor of 2 for vorticity, temperature and surface pressure. Moreover, it can be noticed that forecast errors of temperature and vorticity tend to decrease during the 6 h prediction step. This behaviour was also observed in ensemble Kalman filter assimilation systems, as reported by Mitchell et al. (2002) (Figure 6) and Houtekamer et al. (2005) (Figure 3) for instance. Several possible causes of this constrained spread growth are examined in Hamill and Whitaker (2011).

The addition of a model error term through the inflation of forecast perturbations then leads to a substantial increase of error amplitudes. These results are consistent with earlier studies (Houtekamer et al., 2005), and they indicate that when model error is neglected there is a slight analysis error growth with the model dynamics whereas, when model error is accounted for, error amplitudes grow mostly because of the model error component. On the other hand, the error reduction resulting from the assimilation of new observations is also more pronounced when model error is accounted for in the ensemble. This indicates a stronger analysis effect, which is particularly noticeable for large-scale variables such as temperature (around 15% error reduction) and surface pressure (around 30% error reduction). In the case of humidity, for which no inflation is applied to the background perturbations, the amplitudes of error growth and reduction are almost the same for the two ensemble configurations. However, due to the inflation applied to the other variables, the ensemble spread is increased by around 25%.

In order to examine the analysis effect in more detail, the variance spectra of sample analysis and background errors for temperature near 500 hPa are displayed in Figure 7. Without model error representation, the analysis error is generally equal to or slightly smaller than the background error. At largest scales (approx. up to wavenumber 10), however, the analysis error is larger than the background error. This result may look somewhat surprising since one would expect the analysis to reduce the uncertainty in the background state thanks to the information extracted from the observations. When model error is accounted for in the ensemble, the error reduction is much stronger and the analysis error is smaller than the background error at all scales, in accordance with the expected effect of the analysis step. This difference in the sample variance evolution during the analysis step corresponds to expected effects of the underestimation of background sample variances when model error is omitted in the calculation of background perturbations. Typically, in the case of a perfect model assumption, the reduction of background perturbation amplitudes by the analysis tends to be underestimated compared to the contribution of observation perturbations (see Appendix A). Moreover, when model error is accounted for, the analysis reduces the contribution of synoptic scales more strongly. This can be explained as follows: the synoptic scale's contribution to background error is largely emphasized by the model error simulation on the one hand (Figure 4(d)), and the observations are expected to reduce more efficiently the amplitude of the large-scale part of background error on the other (Daley, 1991, p. 128).

Figure 7.

Horizontal variance spectra of sample analysis error (dashed) and background error (solid) for temperature at a level close to 500 hPa, for (a) a perfect model ensemble and (b) an ensemble with an online adaptive perturbation inflation. Background and analysis errors are calculated at truncation T399. Unit: K2.

Local analysis error reductions are illustrated in Figure 8, which presents maps of the time-averaged ratio of vorticity analysis error with respect to background error for the two ensemble configurations, at a level close to 500 hPa. In both cases, the analysis error reduction is larger over the continental areas of the Northern Hemisphere, where the observation network is dense. The largest error reductions are found over the USA and western and northern Europe, because of a good spatial and temporal coverage of radiosonde observations there. Conversely, the smallest error reductions occur over the tropical oceans (data-poor area). Background error is reduced by 10% on average when model error is accounted for in the ensemble, compared to 5% when model error is neglected. It may also be mentioned that when model error is neglected the magnitude of analysis error can locally be larger than the magnitude of background error (e.g. the Tropics), which is no more the case when model error is simulated.

Figure 8.

Ratio of analysis error standard deviation with respect to background error standard deviation, averaged over the period between 21 January 0000 UTC and 09 February 1800 UTC 2010, for (a) a perfect model ensemble and (b) an ensemble with an online perturbation inflation. The results are for vorticity at a level close to 500 hPa. Since the error reduction is more pronounced in the ‘inflated’ ensemble, each panel has its own colour bar. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

4.4. Local standard deviations

Because of the cycling of inflated background perturbations within the ensemble on the one hand, and because of nonlinearities in the model dynamics on the other, the overall impact of the (uniform) online multiplicative inflation procedure can be expected to be more complex than a simple homogeneous inflation of variance maps. To examine this aspect, variance maps derived from the ‘inflated’ ensemble and from the operational perfect model ensemble (including the offline variance inflation) are compared for the particular date of 2 February 2010 at 2100 UTC.

Figure 9 shows that geographical variations of baseline and new surface pressure standard deviations look broadly similar. Common features, including smaller values over land and larger values over sea, especially in cyclogenesis areas, can be identified. However, when an online perturbation inflation is applied, the contrast between the Tropics and the extra-Tropics tends to be reinforced. Standard deviation values are reduced on average in the Tropics both over land (e.g. South America) and over sea (e.g. Indian Ocean, Pacific Ocean), while they are increased in the extratropical regions. In particular, local variance maxima are locally emphasized in dynamical systems such as lows and troughs (e.g. Southern Hemisphere storm tracks). These local strengthenings are consistent with the enhancement of synoptic scales in the variance spectra (Figure 5(a)). Other features, like the increase of variance in the Siberian anticyclone (relatively data-poor area), reflect the observation network heterogeneity. Similar results are obtained for vorticity (not shown).

Figure 9.

Background error standard deviations of surface pressure corresponding to 02 February 2010 at 2100 UTC, calculated from (a) a perfect model ensemble (including an offline variance inflation) and (b) an ensemble with an online perturbation inflation. Unit: hPa. The mean sea-level surface pressure analysis from the unperturbed system is overlaid, contour: 10 hPa. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Temperature error standard deviations near 850 hPa are displayed in Figure 10. General features such as the positioning of highs and lows are comparable in the two ensembles. However, it can be seen that the representation of model error leads to a reinforcement of data density contrasts between, for example, Europe and Russia. This is related to the stronger analysis effect highlighted in Figures 6(b) and 7(b). Conversely, there is a general increase of error amplitude over the Southern Hemisphere, which can be attributed to the low data density and to the dynamical activity.

Figure 10.

Background error standard deviations of temperature near 850 hPa corresponding to 02 February 2010 at 2100 UTC, calculated from (a) a perfect model ensemble (including an offline variance inflation) and (b) an ensemble with an online perturbation inflation. Unit: K. The mean sea-level surface pressure analysis from the unperturbed system is overlaid, contour: 10 hPa. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

It may also be mentioned that a substantial increase of ensemble variances is observed over the polar regions (e.g. Southern Hemisphere circumpolar ocean) for both surface pressure and temperature. This is likely to reflect combined effects of relatively large perturbation growths in mid- and high-latitude cyclogenic areas (compared to the Tropics) and of the fact that the (inflated) background perturbations are reduced by the analysis to a lesser extent over these relatively data-poor regions (compared to Northern midlatitudes for instance), as indicated by Figure 8(a). This large increase could also be a consequence of the horizontally homogeneous inflation factor applied, to some extent. It would thus be interesting in the future to compare these geographical maps of ensemble spread with innovation-based estimates (as in Berre et al., 2007). Depending on the results, a regionalized version of the inflation could then be considered.

The global multiplicative inflation applied to background perturbations thus has a noticeable impact on ensemble variance fields of all variables (apart from the particular case of specific humidity). The connection with the underlying flow tends to be strengthened, especially for surface pressure and vorticity, as shown by the increase of error variances in the vicinity of intense weather events such as midlatitude storms. On the other hand, the contrast between data-rich and data-poor areas is enhanced for temperature variance, in relation to locally stronger impacts of the data assimilation.

These local variance modifications are likely to have an impact on the quality of analyses and subsequent forecasts. This is investigated in the next section.

5. Impact experiments

In order to investigate the impact of the new ensemble background error statistics on forecast skill, analysis–forecast experiments have been run with the deterministic system. As explained in section 2.1, the Arpège ensemble data assimilation is used to calculate local space- and time-varying variances on the one hand, and global climatological covariance functions on the other. The impact of both these local and global estimates has been studied. The list of experiments and their features are summarized in Table I. Experiment REF uses local variances and global covariances calculated from the baseline perfect model ensemble. Experiment VARlocal uses local variances calculated from an ensemble including an online perturbation inflation, while the global covariances are derived from the baseline perfect model ensemble. Experiment ALL uses local and global statistics derived from an ensemble with an online perturbation inflation.

Table 1. Summary of the experiments performed. Differences between experiments come from the background error statistics used, including both local variances ‘of the day’ (‘Local var’) and global climatological covariance functions (‘Global cov’). ENS PM means that the statistics used are obtained from a perfect model ensemble and ENS INFL means that the statistics used are obtained from an ensemble including an online perturbation inflation.
Exp.Global cov.Local var.
REFENS PMENS PM
VARlocalENS PMENS INFL
ALLENS INFLENS INFL

The experiments were performed with the same configuration as an operational run, over the period between 21 January 0000 UTC and 10 February 0000 UTC 2010 (the first 4 days of the ensemble run are discarded as they correspond to the spin-up period). A wide range of conventional and satellite data is assimilated: surface observations, aircraft data, satellite-derived winds, sea surface observations (e.g. drift buoys, ship reports), in situ sounding data, wind profiler radar data, geostationary satellite winds (atmospheric motion vectors), global positioning system (GPS) ground-based data and radiances from polar-orbiting satellites (e.g. AMSU-A/B, AIRS, SSMI, IASI).

5.1. Impact of local variances

The impact of the new local variance specification can be obtained by comparing forecast skills from the baseline experiment REF, using the perfect model ensemble local variances (with an offline variance inflation), and from the experiment VARlocal, using the new ensemble local variances (with an online perturbation inflation). Both experiments use the same global covariances calculated from the baseline perfect model ensemble.

The impact of the new variances, measured with time-averaged forecast scores (root mean square errors), is neutral to positive on average. Improvements are visible over North America in particular, as shown in Figure 11(a) by the vertical profile of RMS error for 72 h forecasts of geopotential. The corresponding time series at 500 hPa is shown in Figure 11(b), and illustrates that this positive impact is relatively robust in time. Bootstrap tests (Wilks, 2006) indicate that this improvement is significant at the 99% confidence level.

Figure 11.

(a) Vertical profiles of RMS error of 72 h geopotential forecasts (m) for the experiment VARlocal (solid line) and for the baseline experiment REF (dashed). (b) Temporal evolution of RMS error of 72 h forecasts of 500 hPa geopotential (m) for the baseline run REF (dashed line) and for the experimental run VARlocal (solid line). Forecasts are verified against radiosonde observations and scores are shown for a domain covering North America.

5.2. Impact of global covariances

Global covariances estimated from a 3-week ensemble run are used in the 4D-Var assimilation to provide global vertical profiles of standard deviation, homogeneous horizontal correlation functions and non-separable vertical correlations.

The specific impact of the new global covariances, obtained by comparing experiments VARlocal and ALL, is globally neutral (not shown), except for slight positive impacts over a European–Atlantic domain.

These different impacts of the global covariances on the one hand, and of the local variances on the other, look consistent with the associated modifications at play. On the one hand, the new global variances calculated from inflated perturbations are similar to global variances that are inflated offline in the reference system (Eq. (2)), so that their relatively neutral impact is expected to some extent. On the other hand, the use of the new local variances derived from inflated perturbations corresponds to relatively larger modifications in the assimilation system. From this point of view, it is thus consistent to observe that the new local variances have a more significant impact than the new global covariances.

6. Conclusions and perspectives

The aim of this paper was to propose and test a technique to account for model error within the Météo-France variational ensemble assimilation. The choice of a multiplicative inflation procedure has been motivated by its simplicity and its ability to handle all sources of error of unknown origins. The experimental ensemble configuration includes a multiplicative inflation of forecast perturbations after 6 h of model integration, and the inflation factor is adaptively tuned according to a posteriori diagnostics relative to the value of the cost function at its minimum (Talagrand, 1999; Desroziers and Ivanov, 2001).

A first experimentation of this approach showed that the appropriate inflation is around 10% in our current system. The amplification of forecast perturbations by this factor leads to a global increase of ensemble background error standard deviations by approximately a factor of 2 for all model variables, apart from humidity, which is not directly inflated in the present experiments. This increase has been validated by comparison with independent estimates of background errors derived from the Desroziers et al. (2005) diagnostics. The impact of the model error simulation on background error is particularly noticeable at synoptic scales, which correspond in particular to low-predictability systems such as midlatitude storms and tropical cyclones. Consequently, the representation of model error tends to strengthen the connection of local errors with the synoptic conditions, as indicated by larger values of surface pressure and vorticity variances in dynamically active regions, for instance.

The evolution of analysis and background errors in a data assimilation cycle is also markedly modified. It is first observed that analysis error grows mostly because of the model error term. The analysis effect is also more pronounced, especially for large-scale fields such as temperature and surface pressure. This contributes to reinforce the contrast between data-rich and data-poor areas in variance maps of temperature in particular.

Ensemble-estimated variances are used routinely in the 4D-Var assimilation of the global Arpège model to introduce flow-dependent information in the background error covariances. Replacing the current perfect model statistics (including an offline variance inflation) by the new statistics computed from inflated background perturbations has a neutral to positive impact on forecast scores.

It may be mentioned that such estimates of model error variances, here applied in the context of a variational ensemble assimilation, can also be useful for other assimilation algorithms involving a model error term such as weak-constraint 4D-Var (Trémolet, 2007) and EnKF, for instance.

Moreover, while a global inflation factor has been used in this study, a regionalized application of the diagnostics and of the inflation will be interesting to consider in the future.

Appendix

A1. Evolution of sample variance during the analysis step

The evolution of analysis and background errors has been examined in section 4.3. Theoretical justifications of the observed evolution during the analysis step are given in this section.

In an ensemble of perturbed assimilations, the analysis perturbations are related to background and observation perturbations by the following equation (Berre et al., 2007):

equation image(10)

It can be shown from Eq. (10) that the sample analysis covariances are given by

equation image

where equation image and equation image are the sample background and observation covariances, respectively (as detailed for equation image in section 2.2).

The impact of background perturbation amplitude (depending on whether model error is represented or neglected) on the representation of the analysis effect is detailed below.

A2. Case 1: the amplitude of equation image is consistent with specified background error variances

When the amplitude of background perturbations is consistent with specified background error variances (equation image), the sample analysis covariances reduce to (Burgers et al., 1998, Eq. (10))

equation image

This means that the analysis sample variance is smaller than the background sample variance (equation image. This behaviour is observed when model error is accounted for in the ensemble (Figure 7(b)).

A3. Case 2: the amplitude of equation image is severely underestimated

When background perturbations are severely underestimated and thus are relatively close to zero (equation image), then equation image. Then equation image, which implies that the analysis sample variance is larger than the background sample variance equation image since equation image.

A4. Case 3: the amplitude of equation image is underestimated by a factor of 2

The case with background perturbations underestimated by a factor of 2 (equation image) is likely to be intermediate between the two cases above. This corresponds to the underestimation observed in a perfect model framework (section 4.1).

Under the simplifying assumptions that background errors and observation errors are the same order of magnitude, i.e. RB, and that the observation operator is the identity, the Kalman gain matrix is given by KI/2. Moreover, since equation image it follows that

equation image

This situation thus implies that the analysis sample variance is relatively close to the background sample variance equation image, as observed in Figure 7(a).

Ancillary