To account for uncertainties in both the initial conditions and forecast model and to combat the associated forecast error and flow-dependent predictability, ensemble methods have become popular for producing numerical weather forecasts (Molteni and Palmer, 1993; Toth and Kalnay, 1993) and for performing data assimilation (Evensen, 1994). The idea behind ensemble-based methods is that the nonlinear chaotic dynamics of the underlying forecast model and the associated sensitivity to initial conditions cause an ensemble of trajectories to explore sufficiently large parts of the phase space in order to deduce meaningful statistical properties of the dynamics. However, all currently operational ensemble systems are underdispersive, meaning that the true atmospheric state is on average outside the statistically expected range of the forecast or analysis (e.g. Buizza et al., 2005; Hamill and Whitaker, 2011). This detrimental underdispersiveness can be linked to two separate sorts of model error: a dynamical model error due to a misrepresentation of the physics of the forecast model and a numerical model error due to the choice of the numerical method used to simulate those forecast models.
It has been argued (Palmer, 2001) that dynamical model error is largely due to a misrepresentation of unresolved subgrid-scale processes. The very active field of stochastic parametrization is aimed at reintroducing statistical fluctuations of the unresolved degrees of freedom and a direct coupling between the resolved and unresolved degrees of freedom into the equations of motion (Palmer and Williams, 2010, provide a recent review on current trends). Indeed, using a low-dimensional toy model, Mitchell and Gottwald (2012) showed that stochastic parametrizations can lead to superior skill performance in ensemble data assimilation schemes in situations where the dynamics involve rapid large-amplitude excursions such as in regime switches or in intermittent dynamics.
Numerical model error is often introduced deliberately to control numerical instabilities arising in the simulation of geophysical fluids, such as in numerical weather prediction or climate dynamics. In such simulations, numerical codes tend to produce large errors linked to numerical instabilities at the grid resolution level, which limits the reliability of the forecast over the forecast window. In a data assimilation context, this numerical ‘noise’ can lead to an unrealistic overestimation of forecast-error covariances at small scales. The standard approach to dealing with these numerical instabilities is to add various forms of artificial dissipation to the model (Durran, 1999). However this has several well-documented drawbacks. In the context of ensemble data assimilation, artificial viscosity diminishes the spread of the forecast ensemble, causing detrimental underestimation of the error covariances (Burgers et al., 1998; Houtekamer and Mitchell, 1998; Anderson and Anderson, 1999; Houtekamer et al., 2005; Charron et al., 2010). Most notably, artificial dissipation implies unrealistic and excessive drainage of energy out of the system (Shutts, 2005), effecting processes ranging from frontogenesis to large-scale energy budgets. For example, Palmer (2001) reports on the unphysical dissipation of kinetic turbulent energy caused by mountain gravity-wave drag parametrizations. Skamarock and Klemp (1992) introduce divergence damping to control unwanted gravity wave activity in hydrostatic and non-hydrostatic elastic equations to stabilize the numerical scheme, leading to numerical energy dissipation (also Durran, 1999). Blumen (1990) controls the collapse of near-surface frontal width by artificially introducing momentum diffusion in a semi-geostrophic model of frontogenesis, with the effect of producing unrealistic energy dissipation rates. In semi-Lagrangian advection schemes, Côté and Staniforth (1988) and Ritchie (1988) find unrealistic energy dissipation rates that are caused by interpolation. Besides affecting the small-scale dynamics, numerical dissipation also has a major effect on the large scales and their energy spectra (Palmer, 2001; Shutts, 2005). In ideal fluids, which often serve as a good approximation of the large-scale dynamics (Salmon, 1998), numerical dissipation destroys dynamically important conservation laws which may be undesirable in long time simulations such as in climate modelling (Thuburn, 2008). Using statistical mechanics, Dubinkina and Frank (2007), Dubinkina and Frank (2010) show how the choice of the conservation laws respected by a numerical scheme affects the statistics of the large-scale fields. There exists a plethora of methods to remedy the unphysical energy loss due to artificial diffusion and reinject energy back into the numerical model, for example via energy backscattering (Frederiksen and Davies, 1997; Shutts, 2005) or systematic stochastic model reductions (Majda et al., 1999).
Here we will look at these issues in the context of ensemble data assimilation. Rather than controlling numerical instabilities of the dynamic core via appropriate discretization schemes of the numerical forecast model, we will modifiy the actual ensemble data assimilation procedure, avoiding excessive numerical dissipation. Underdamping in dynamic cores causes the ensemble to acquire unrealistically large spread. This overdispersion is exacerbated in sparse observational networks where, for the unobserved variables, unrealistically large forecast covariances are poorly controlled by observational data (Liu et al., 2008; Whitaker et al., 2009). The questions we address in this article are: (i) Can one use numerical forecasting models which are not artificially damped, yet still control the resulting overestimation of forecast covariances within the data assimilation procedure? (ii) Is it possible to use the increased spread induced by numerical instabilities in a controlled fashion to improve the skill of ensemble data assimilation schemes?
In a recent article, Gottwald et al. (2011) proposed a variance-limiting ensemble Kalman filter (VLKF) which controls overestimation in sparse observational networks by including climatological information of the unobserved variables. In a perfect model data assimilation scheme, unrealistic overestimation of the error covariances occurs in sparse observational networks as a finite size effect; it was shown that, for short observational intervals of up to 10 h, the VLKF produces superior skill when compared to the standard ensemble transform Kalman filter (ETKF) (Bishop et al., 2001; Tippett et al., 2003; Wang et al., 2004). Here we will apply the VLKF in the case of an imperfect, underdamped forecast model, and for large observation intervals. Unlike in the perfect model case where for large observation intervals the forecast-error covariances approach the climatic variance, when forecasting with underdamped forecast models the error covariances will be larger than the climatic variance. This will be controlled by the VLKF, invoking a weak constraint onto the mean and the variance of the unobserved variables, leading to an improvement in the analysis skill. We will study this effect using the Lorenz-96 model (Lorenz, 1996), where we assume that the truth is evolving to the standard set of parameters. To investigate whether underdamped models can be used as forecast models, we then use forecast models with smaller values of the linear damping term of the Lorenz-96 system to generate ensembles with forecast spreads exceeding the true climatology.
Our main result is that VLKF produces significantly better skill than ETKF, increasing with the strength of the model error, the sparsity of the observational network and the observational noise covariance. We will establish that a variance-limiting weak constraint may be used as an adaptive alternative to artificial viscosity damping, counteracting model error ‘on the fly’ when underdamping of the numerical forecast model causes an unrealistic overestimation of the forecast ensemble. However, we will see that there is a trade-off since for large model error the skill of VLKF becomes comparable to the skill of an analysis involving no forecast but using an analysis comprised of the observations and the climatological mean for the observed and unobserved variables, respectively. It will be shown that this is important for large observation times when the analysis is not tracking anymore due either to model error or to the chaotic nature of the underlying dynamics, and one can only improve the analysis, assuring a reliable reproduction of statistical properties of the dynamics.
In the next section we briefly describe the VLKF. In section 3, we introduce the Lorenz-96 model and describe the underdamped forecast model used. In section 4, we present results showing under what conditions the VLKF produces better skill than the usual ETKF. We conclude with a discussion and outlook in section 5.
2. The variance-limiting Kalman filter
The VLKF is a variation of the standard ensemble Kalman filter designed to control overestimation of error covariances caused by finite ensemble sizes in sparse observational grids (Gottwald et al., 2011). Whereas usually only direct observations and model forecasts are combined to produce an analysis in data assimilation, it was proposed to incorporate climatological information to fill the data voids in sparse observational grids. In the atmospheric sciences, data voids are a huge obstruction to climate reanalyses (Bengtsson et al., 2007; Compo et al., 2011; Whitaker et al., 2004) when assimilating data from pre-radiosonde era, or when attempting to include the mesosphere in numerical weather forecasts where the energetics are dominated by unobserved gravity waves (Eckermann et al., 2009; Polavarapu et al., 2005; Sankey et al., 2007). Climatic information is used to limit the posterior error covariance of the unobserved variables so as not to exceed their climatic covariance. It was shown that this limitation of the analysis-error covariance controlled undesirable overestimation of error covariances which are caused by the finite size of the forecast ensembles. Furthermore, the VLKF drives the mean of the unobserved variables towards the climatological mean, significantly increasing the analysis skill. We remark that one may use other values than the climatic covariance to control the analysis error covariance if one interprets the variance constraint as a numerical tool only to control overestimation of the error covariances. We present a brief summary of the algorithm here. For a detailed derivation the reader is referred to Gottwald et al. (2011).
Given an N-dimensional dynamical system
which is observed at discrete times ti = iΔtobs, data assimilation aims at producing the best estimate of the current state given a typically chaotic, possibly inaccurate model ) and noisy observations of the true state zt (Kalnay, 2002). We assume that the state space is decomposable according to z = (x,y) with observables, for which direct observations are available, and pseudo-observables, for which only some integrated or statistical climatological information is available. Here we will assume n + m = N, however this is not essential.
Observations are expressed as a perturbed truth according to
where the observation operator maps from the whole space into observation space, and is assumed to be independent identically distributed observational Gaussian noise with associated error covariance matrix Ro.
We further introduce an operator which maps from the whole space into the space of the pseudo-observables spanned by y. We assume that the pseudo-observables have variance and constant mean . This is the only information available for the pseudo-observables, and may be estimated, for example, from historical data or long time simulations. The error covariance of those pseudo-observations is denoted by .
In an ensemble Kalman filter (EnKF) (Evensen, 2006) an ensemble with k members zk
is propagated by the model dynamics according to
This forecast ensemble is split into its mean and ensemble deviation matrix Z′f. The ensemble deviation matrix Z′f can be used to approximate the ensemble forecast covariance matrix via
Note that Pf(t) is rank-deficient for k < N which is the typical situation in numerical weather prediction where N is of the order of 109 and k of the order of 100.
Given the forecast mean , the forecast-error covariance Pf and an observation yo, an analysis mean is given by
with analysis-error covariance matrix
The variance of the unresolved pseudo-observables Hwz is controlled by setting
This yields upon using the Sherman–Morrison–Woodbury formula (e.g. Golub and Loan, 1996) the equation for the as yet unspecified error covariance matrix for the pseudo-observables Rw
is the analysis-error covariance of an equivalent EnKF without variance constraint. We remark that the matrix Rw itself is never required in the VLKF algorithm, so an expensive inversion of is not explicitly needed. Furthermore, the matrix can also be calculated without computing inverses of the forecast-covariance matrix by means of the Sherman–Morrison–Woodbury formula. It is readily seen that for sufficiently small background-error covariance Pf, the error covariance Rw is not positive semi-definite. To ensure that the error covariance is non-negative, we diagonalize ; introducing with for Dii ≥ 0 and for Dii < 0, we set . Hence the variance limiter acts only on those eigendirections of whose corresponding singular eigenvalues are larger than those of the prescribed (climatic) covariance Aclim. The diagonalisation of poses an additional computational complexity when compared to ETKF.
To determine an ensemble Za which is consistent with the analysis-error covariance Pa and satisfies
we use the method of ensemble square root filters (Simon, 2006). In particular we use the ETKF (Bishop et al., 2001; Tippett et al., 2003; Wang et al., 2004), which seeks a transformation such that the analysis deviation ensemble is given as a deterministic perturbation of the forecast ensemble Zf via . Details of the implementation can be found in Bishop et al. (2001), Tippett et al. (2003) and Wang et al. (2004). Alternatively one could choose the ensemble adjustment filter (Anderson, 2001) in which the ensemble deviation matrix is pre-multiplied with an appropriately determined matrix . Yet another method to construct analysis deviations was proposed in Bergemann et al. (2009) where the continuous Kalman–Bucy filter is used to calculate without using any computations of matrix inverses, which may be advantageous in high-dimensional systems. A new forecast is obtained by propagating Za with the nonlinear forecast model to the next observation time, where a new analysis cycle will be started.
In the perfect model case, when Aclim and aclim are given by the climatology, the forecast model will acquire comparable forecast-error covariance Pf for large observational intervals Δtobs using either the ETKF or VLKF. Hence both filters will have approximately the same input parameters prior to the analysis step and, as was shown in Gottwald et al. (2011), the filters perform equally well. This is not true for imperfect models with an underdamping forecast model; when the variance-limiting constraint is switched on, the background covariance of the pseudo-observables is set to Aclim = σclim2I for the VLKF, much smaller than that for the ETKF when using an underdamped model. Note that this implies that the forecast-error covariance is also typically much smaller for the VLKF than ETKF over short observation intervals.
3. The Lorenz-96 model
Consider the following modification of the Lorenz-96 model (Lorenz, 1996)
with z = (z1,...,zD)T and periodic zj = zj+D, where we have introduced a variable linear damping parameter γ. The Lorenz-96 model is a standard test bed for data assimilation (e.g. Ott et al., 2004; Fisher et al., 2005) as it is computationally manageable but still incorporates crucial ingredients of real midlatitude atmospheric flows such as nonlinear energy conservation, advection, forcing and linear damping. We set the number of degrees of freedom as D = 40 and F = 8 for the forcing. For γ = 1, the classical value of the damping in the Lorenz-96 system, this corresponds to a time-scaling of 5 days per model time unit, an e-folding time of 2.1 days and a distance between adjacent grid points of 750 km, roughly equalling the Rossby radius of deformation (Lorenz, 1996).
Figure 1 illustrates how the climatic mean μclim and variance depend on the value of the damping parameter γ. The climatic values are obtained from simulations of one long time trajectory (1000 days). Note that in the Lorenz-96 system all variables zj share the same statistical behaviour. Furthermore, we show the variances σ2 when calculated from simulations running only over a finite length of time, such as analysis cycle length. As expected, the variance increases as the dynamics get less and less damped for a fixed length of time, and increase with the simulation length Δtobs for fixed damping parameter γ (asymptoting to the climatic variance).
Two characteristic time-scales of the Lorenz-96 system (4) will be important in the following: the e-folding time, measured here by the inverse of maximal Lyapunov exponent λmax, and the advection time Tadv, which is a measure for the time taken for a front to propagate from one site j to the next j + 1. In Figure 2, we show how these characteristic time-scales vary with the damping parameter γ. Firstly, we show the inverse of the maximal Lyapunov exponent λmax, which is a measure for the predictability time of the Lorenz-96 dynamics and correlates with the decorrelation time. For the original Lorenz-96 model with γ = 1, we reproduce the e-folding time of 50 h, i.e. 2.1 days. In accordance with the observed increase in variance, the predictability time decreases with decreasing damping. As a second characteristic time-scale, we show the advective time Tadv. Since the nonlinear term in (4) can be interpreted as a finite-difference discretization of an advection term zzx, information from one grid point j is mainly transported to its right neighbours in an asymmetric fashion. We estimate Tadv by initializing an integration of the Lorenz-96 model with only one non-zero variable and then measuring the time taken for the resulting front to propagate across sites. The characteristic time-scale of this information transport is important when discussing the influence of the sparsity of an observational network on the filtering skill. Figure 2 shows that this time-scale decreases with increasing model error.
4. Numerical results
We now investigate how and when the VLKF can be used to control the increased covariances of an underdamped forecast model. In particular, we will study the situation of a sparse observational grid.
We will address two issues. First, we will investigate under what conditions the VLKF as described in section 2 produces better analysis skill than the ETKF. Second, we will investigate when the VLKF produces better skill than a worst-case analysis which does not involve any forecast but simply constructs the analysis by taking the observational data and the climatic mean for the observed and the unobserved analysis variables, respectively (a ‘poor-man's analysis’). This latter unfortunate situation can occur, for example, for moderate observation intervals Δtobs, as we will see. Besides the degree of underdamping γ, the skill will be seen to depend on the sparsity of the observational grid used as well as on the observation interval and the observational noise level.
The main result will be that the VLKF framework can indeed provide skill improvement over the standard ETKF. However, as the level of model damping is decreased, we find that there is a trade-off between increasing improvement over ETKF on the one hand, and decreasing improvement over the poor-man's analysis on the other. The skill improvement can be explained by the improved ability of the VLKF to ensure the correct statistical properties of the analysis, as opposed to pathwise tracking of the truth, for small forecast model damping. We find analogous results as the observational sparsity Nobs or the magnitude of the observational noise is increased.
We generate a truth zt from a realisation of the Lorenz-96 system (4) with γ = 1. Observations yo are then generated for the observables Hz with Gaussian error covariance Ro = (ησclim)2I, and are available after each observation interval Δtobs at every Nobsth site. The observations are then assimilated in an ensemble filter framework using the Lorenz-96 model (4) with damping parameter γ < 1 as forecast model, mimicking the situation of underdamped forecast models. We perform twin experiments where the same truth and observations are used for data assimilation with both the VLKF and ETKF. Both filters are run with k = 41 ensemble members and the same forecast model (4) with γ < 1, making the forecasts underdamped with respect to the truth zt. Both filters are spun up for 100 cycles to ensure statistical stationarity of the ensembles. We employ 5% constant global multiplicative covariance inflation in both filters.
We use an implicit mid-point rule (e.g. Leimkuhler and Reich, 2005) to simulate the system (4) to a time T = 60 time units with a time step dt = 1/240. The total integration time corresponds to 300 days, and the integration timestep dt corresponds to half an hour. The climatic mean and standard deviation over all variables with γ = 1 are estimated as temporal averages from a long trajectory as μclim = 2.34 and σclim = 3.63 respectively, which determines our true climatology.
In Figures 3 and 4, we show sample analyses using both filters, ETKF and VLKF, with Nobs = 4 and damping factor γ = 0.5 for both Δtobs = 24 h and Δtobs = 48 h, exemplifying moderate and large observation intervals respectively. We show analyses for an arbitrary unobserved component and an arbitrary observed component of the Lorenz-96 model. Both filters track the truth for the observed components reasonably well for both moderate and large observation intervals, with the analysis tending more towards the observations for large Δtobs. However, for the pseudo-observables with moderate observation interval Δtobs = 24 h, the ETKF tracks the truth better than the VLKF despite the inclusion of additional climatological information. For large observation intervals Δtobs = 48 h, neither filter tracks the truth for the pseudo-observables well. However, the additional input of the climatological information into the VLKF leads to an analysis with statistics which better resemble the true climatology, thereby improving the overall RMS errors and skill. It is also clearly seen that the main contribution to the error stems from the pseudo-observables.
Figures 3 and 4 suggest that the notion of what constitutes a good performance of a filter depends on the length Δtobs of the observation interval. When aiming for improved performance in the case of moderate observation intervals, one is interested in tracking the truth for both the observables and pseudo-observables. In the case of large observation intervals, one can only expect to track the truth for the observed variables and produce analyses which exhibit the same statistical behaviour as the truth for the pseudo-observables. We remark that, for very large observational sparsity Nobs, tracking cannot be demanded even for moderate observation intervals; for example with Δtobs = 6 h the analysis of an observational grid with Nobs = 20 does not produce tracking analyses for any variable zj.
In Figure 5 we show the average maximal singular values sH and of the forecast- and analysis-error covariances projected onto the observation and pseudo-observation subspaces respectively. As expected, the covariances increase with increasing model error, as well as with increasing sparsity. The variance constraint of the VLKF markedly controls the analysis covariance of the unobserved variables to always assume values below the climatic variance. This causes the forecast covariances to be limited as well, albeit at values greater than the climatic variance . For large observational sparsity Nobs, the analysis covariance of the unobserved variables is set to the climatic variance for the VLKF, whereas for the ETKF the analysis covariances grow larger than with increasing sparsity. Interestingly, the VLKF also limits the covariances for the observed variables. As we will see, this may have negative effects and may lead to an underdispersive ensemble which can spoil the VLKF analysis for the observed variables.
In Figure 6 we show how for long observation intervals Δtobs = 48 h the true climatic mean and variance are better preserved by the VLKF analysis than by the ETKF analysis. The true climatic mean μclim is reasonably well preserved by the VLKF analysis whereas the ETKF greatly underestimates it. The VLKF analysis underestimates the true climatic variance , but not as severely as the ETKF does. We will see that this leads to a marked increase in skill, especially for large model error γ < 1.
The fact that the variance of the analysis decreases with decreasing damping may seem counterintuitive at first sight. This can be justified by considering a simple two-dimensional system where x is observed and y is unobserved. The Kalman analysis update (cf. Eq. (2)) for the unobserved variables is
where Pf,xx > 0 and Pf,xy measures the forecast-error covariance between the unobserved and observed variable. Note that, due to finite size ensemble sizes, one can have Pf,xy≠0 even if dynamically there are no correlations between observed and unobserved variables. We argue that the decrease in variance of the analysis with decreasing damping is due to the analysis being damped stronger for smaller values of γ in the case of the Lorenz-96 system (4). For the simple system this is the case when
on average, provided that also on average. Since the climatic mean is positive and decreases with decreasing damping γ (cf. Figure 1), and on average. Moreover, as all variables zj are statistically equal due to the periodicity of the Lorenz-96 system, the forecast-error covariances between observed and unobserved variables HPfHTw (equivalent to Pf,xy in the simple model) must also be negative on average. Hence the variance of the analysis time series has the same decreasing tendency with decreasing damping γ as the climatic mean.
We now quantify the performance of the VLKF and the standard ETKF by the site-averaged RMS error
between the truth zt and the ensemble mean , where L = ⌊T/Δtobs⌋ is the number of analysis cycles and DG denotes the number of variables involved. We introduced the norm to investigate the error over all variables ℰ using G = I, the error ℰo distributed over the observed variables using G = HTH, and the error ℰuo distributed over the unobserved variables using G = HTwHw. We also introduce a measure to quantify how well a filter performs against the poor-man's analysis of using the observations and the climatic mean for the observed and unobserved components of the analysis respectively, implying and . Assuming that and are uncorrelated and setting Aclim = σclim2Im, we can calculate the RMS error in the poor-man's analysis as
where L = ⌊T/Δtobs⌋ is the number of analysis cycles as in (5), giving the RMS error
with Ro = (ησclim)2I.
To quantify the proportional improvement in RMS errors for the VLKF over ETKF we introduce the analysis skill
where the RMS errors ℰ are defined by (5). Values of indicate an improvement of the VLKF over ETKF. We also define the corresponding skills for proportional improvement of the ETKF and VLKF over the poor man's analysis
We first investigate the dependence of skill upon the observation interval Δtobs. In Figure 7 the skills , and are shown as a function of the model error γ for different observation intervals Δtobs with fixed sparsity Nobs = 4. The figures reveal an intricate dependence of the skill on both the observation interval and the model error. We have checked that for Δtobs > 2.1 days, the e-folding time for the Lorenz-96 model with γ = 1, the skill curves collapse onto one curve, independent of the observation interval. This indicates that the forecast-error covariance, which saturates for large observational times to the climatic covariance, is the main factor driving filter performance.
We will treat the near-perfect model case γ close to one and the strong model error case with small γ separately. The perfect model case with γ = 1 was already considered in Gottwald et al. (2011). We confirm here that the VLKF is superior to the standard ETKF, its skill increasing with decreasing observation intervals Δtobs. For smaller observation intervals, skill improvements of more than 20% can be achieved, and the analysis of the pseudo-observables track the truth surprisingly well (cf. Gottwald et al., 2011). Note that both filters perform better than the poor man's analysis with and for γ = 1. The variance-limiting filter here successfully controls the overestimation of the analysis-error covariance caused by the combination of finite ensemble sizes and sparse observations.
The case of model error with γ significantly smaller than 1 – our main objective in this work – shows an interesting dependence on both the observation interval and the model error. For the discussion below, it is important to note that, for sufficiently large model error, the analysis-error covariances projected onto the pseudo-observables exceed the climatic variance ; hence the background-error covariance matrices which are used to initialize an analysis cycle are larger than for the ETKF. On the other hand, the VLKF limits the analysis error covariance (projected onto the pseudo-observables) from above to , independent of the observation interval.
It is seen in Figure 7 that for moderate observation intervals Δtobs (24 h), the VLKF performs worse than the ETKF with skills < 1 decreasing with increasing model error (i.e. decreasing values of γ). On the contrary, for large observation intervals with Δtobs > 36 h, the skill > 1 increases with increasing model error. Furthermore, for moderate observation intervals the VLKF performs increasingly worse than the poor man's analysis with increasing model error, but outperforms it for all values of γ for large observation intervals. The ETKF, however, always performs worse than the poor man's analysis for all observation intervals in the presence of sufficiently large model error. We first explain the behaviour of the skill observed for moderate observation intervals and then for large observation intervals.
Over moderate observation intervals, the forecast ensemble acquires only little spread, as already encountered in Figure 1. This small spread in conjunction with model error can lead to instances where the analysis mean performs large excursions away from the truth, thereby spoiling the analysis. This effect is stronger for the VLKF as it is initialized with a smaller ensemble spread associated with the constrained analysis-error covariance as discussed above. Increasing the model error by decreasing γ increases the probability of the ensemble mean diverging from the truth, thereby decreasing the skill. This explains the poor performance of the VLKF over ETKF with < 1 for moderate Δtobs, and the poor performance of both the ETKF and VLKF over the poor man's analysis with and , as illustrated in Figure 7.
For large observation intervals the situation is different. Large observation intervals are characterized by the forecast dynamics having sufficiently decorrelated so that the forecast ensemble will have equilibrated with (γ-dependent) covariance, having lost memory of the background initialization. This has the effect that both filters will acquire comparable forecast-error covariance Pf over the forecast interval Δtobs. The large spread which is acquired by the forecast ensemble during large observation intervals prohibits the analysed pseudo-observables tracking the truth well (cf. Figure 4). Skill improvements can only be achieved by (i) tracking the truth for the observed variables as guided by the observations, and (ii) assuring the correct climatological behaviour for the pseudo-observables. This latter point is where the variance-limiting method is beneficial, as illustrated in Figure 6. The more accurate statistical properties of the VLKF analysis results in improved skill with > 1 for all values of γ, as shown in Figure 7.
However, Figure 7 also shows that the increased skill for large observation intervals with increasing model error comes at the price of the VLKF becoming less able to outperform the poor man's analysis with error . As expected, asymptotes to 1 from above for decreasing values of γ, whereas becomes less than 1 for sufficiently large model error. We found that the VLKF performs better than the poor man's analysis for all values of γ for Δtobs > 36 h.
We conclude that the VLKF may be beneficial for ensemble-based data assimilation with large underdamping model error and large observation intervals Δtobs. For smaller observation intervals Δtobs < 18 h with sufficient model error, the standard ETKF performs better than the VLKF (not shown). However, for small Δtobs with a near-perfect model (γ ≈ 1), it is beneficial to use the VLKF, as shown in Gottwald et al. (2011). For large observation intervals, the overall skill increases with increasing model error γ due to the imposed control of the increased error covariances. However, there is a trade-off between increased skill improvement of the VLKF over ETKF, and of the skill improvement of the VLKF over the poor man's analysis.
We now look at the dependence of the skill on the sparsity of the observational grid Nobs in the presence of model error. Numerical simulations show that the critical observation interval above which we observe skill improvement (with > 1 for all values of γ) decreases with increasing sparsity Nobs. Note that this impacts on the notion of what is considered to be moderate or large observation interval in the discussion above. This dependence of skill on observational sparsity can be explained by noticing that sparser observations imply an increased forecast-error covariance and ensemble spread for a given observation interval, as illustrated in Figure 5.
We have already seen that the main contribution to the overall RMS error stems from the pseudo-observables. Therefore one may be tempted to conclude that skill improvement consistently gets better for increasing Nobs. This is correct when only comparing the VLKF and ETKF. However, as in the case of increasing model error for fixed sparsity discussed above, the increased sparsity for fixed model error causes the VLKF to be less effective in outperforming the poor-man's analysis. Figure 8 illustrates how increased sparsity affects the overall RMS error ℰ. It is seen that the ETKF increasingly performs worse than the poor-man's analysis with increasing model error and increasing Nobs. In contrast, the RMS errors for the VLKF are below the RMS error of the poor man's analysis for all values of γ, and they asymptote to for large sparsity and large model error, in accordance with our observations in Figure 5 on the singular values of the error covariances of the VLKF. The trade-off between gain in skill of the VLKF over ETKF and less good skill with respect to the poor-man's analysis for increasing sparsity Nobs can be explained as follows. For sparse observations, information from the observables cannot propagate to distant pseudo-observables within the observation interval Δtobs (cf. Figure 2). This causes the forecast and analysis for those pseudo-observables to be essentially random variables with temporal mean μclim and variance for the VLKF, and μ(γ) and σ2(γ) for the ETKF. Hence increased sparsity leads to ℰV asymptoting from below to for all values of γ, with increasingly worse performance for decreasing damping γ. In contrast, for the ETKF for sufficiently large sparsity, for all values of γ.
It is illuminating to see how the RMS errors are distributed over the observed variables and the unobserved variables. In Figure 9 we depict the RMS errors ℰo of the observed variables and ℰuo of the unobserved variables, as defined in (5). We restrict our discussion to large observation intervals Δtobs = 48 h. Consistent with our observation that the overall skill improvement is dominated by the superior analysis of the pseudo-observables, ℰuo exhibits the same behaviour and trends as the overall RMS error ℰ (cf. Figures 7 and 8). The error for the observed variables shows different behaviour. It is seen that the standard ETKF produces better or equal RMS errors than the VLKF for all values of the model error γ. The inferior performance of the VLKF over ETKF for the observed variables is an instance of underestimation of the error covariances – the familiar situation in ensemble data assimilation. As seen in Figure 5, where we show the average maximal singular value sH of HPaHT for the ETKF and VLKF, the variance constraint of the VLKF for the pseudo-observations also significantly decreases the covariance in the observed subspace. The observed poor performance of the VLKF for the observed variables suggests that the limited covariance in the observed subspace is insufficient to provide enough ensemble spread for a reliable analysis.
For small model error, both filters produce worse analyses than an analysis consisting of observations only. This is again linked to insufficient spread of the forecast ensembles. We have confirmed that for larger ensemble sizes for all γ with both the ETKF and VLKF. Underestimation of error covariances due to finite size effects in ensemble filters are usually controlled by some form of covariance inflation, such as multiplicative inflation whereby the prior forecast-error covariance is multiplied by an inflation factor δ > 1 (Anderson and Anderson, 1999). In Figure 10 we show how the RMS error of the observed variables ℰo decreases below the RMS error of the observed variables for γ = 1 and increasing inflation factor δ. We note that, for very large values of δ in the perfect model situation, the ETKF and VLKF produce analyses of equal RMS error. Increasing the forecast covariance inflation has the same effect as increasing the model error (i.e. decreasing γ) in producing larger forecast ensemble spread.
Further, it is seen in Figure 9 that, for sufficiently large model error, the RMS errors of the observables ℰo for the VLKF and ETKF converge to the same γ-independent value . This can be explained by the greater inflation for less damped forecast models; less model damping increases the forecast-error covariance Pf, and so the analysis tends to trust the observations more. As such, as γ → 0 for both filters, albeit slightly reduced by the Kalman filtering. Due to the variance constraint, the RMS error of the observed variables ℰo asymptotes to at larger model error for a given observational sparsity.
The value of the model damping γ below which the forecast variance is sufficiently large for the filter to trust the observations depends on the sparsity Nobs. The larger the sparsity Nobs, the larger the value of γ for which the RMS error ℰo of the two filters collapses onto the constant value . For Nobs = 8, for all values of γ for the ETKF, and ℰo assumes this value at roughly γ < 0.7 for the VLKF. For Nobs = 3, for γ < 0.7 for the ETKF and for γ < 0.25 for the VLKF. This is linked again to the increased forecast-error covariances caused by the sparsity, as illustrated in Figure 5. Figure 5(a) shows that, for a given observation interval and model error γ, the forecast-error covariance increases with sparsity Nobs, causing the observations to be weighted more by the analysis.
We remark that, while for the observed variables there is generally little or no improvement in the RMS error of the VLKF over ETKF, the VLKF still creates more reliable ensembles, as characterised by the ranked probability histogram (also known as Talagrand diagram) (Anderson, 1996; Hamill and Colucci, 1997; Talagrand et al., 1997). In such a histogram, the forecast ensemble is sorted Xf = [xf,1,xf,2,...,xf,k] and bins (−∞,xf,1), (xf,1,xf,2), ... , (xf,k,∞) are generated at each forecast step. We then increment whichever bin the truth falls into at each forecast step to produce a histogram of probabilities Pi of the truth being in bin i. A reliable ensemble is considered to be one where the truth and the ensemble members can be viewed as drawn from the same distribution. A flat probability density histogram therefore is seen as indicating a reliable ensemble for which each ensemble member has equal probability of being nearest to the truth. A convex probability density histogram indicates a lack of spread of the ensemble, while a concave diagram indicates an ensemble which exhibits too large a spread. In Figure 11 we show ranked probability histograms for an observed variable with γ = 0.5, and all other parameters as in Figure 9. The figure shows that for Nobs = 3, 4 and 8 the VLKF makes the forecast ensemble more reliable by controlling the spread.
In Figure 12 we show how the overall skill, as well as the skills and , vary with the observational noise η, where Ro = (ησclim)2I. The VLKF outperforms both the ETKF and the observations/climatology for large enough noise η ≥ 0.25. Large observational noise gives preference to the pseudo-observables, whose unrealistically large error covariances as seen in Figure 5 are limited by the VLKF. This is particularly true for large model error. It is seen that, for sufficiently large observational noise η ≥ 0.25, the VLKF performs better than a poor man's analysis for all values of the model error γ.
Our numerical results suggest that including climatological information of the mean and variance of the unobserved variables in an ensemble filter can be beneficial in the case when the forecast-error covariances are overestimated. In particular, this occurs when sparse observations are taken at large observation intervals and the forecast model is sufficiently underdamped. However, we have also encountered situations where the additional variance constraint provided by the VLKF causes error covariances to be underestimated, spoiling the analysis (albeit only for the observables and not, as has to be stressed, for the overall RMS error). This is the case for moderate observation intervals with large model error, and also for large observation intervals in the case of near-perfect models, but only if one is interested in the analysis of just the observables. This latter situation is not particularly limiting, as RMS errors for the observables are relatively small compared to those for pseudo-observables, and the finite size effects can be eliminated here by basic multiplicative covariance inflation. For sufficiently large observational noise levels, when the analysis puts more weight on the forecast, controlling the unrealistic overestimation caused by underdamping model error using the VLKF becomes the more effective strategy.
We argue that model error associated with an underdamped forecast model leads to an increased forecast-error covariance and increased associated ensemble spread by the same mechanism by which covariance inflation is beneficial. The variance-limiting filter controls this increased spread and keeps it within climatologically reasonable bounds, thereby leading to a lower RMS error and improving the overall skill.
We remark that one may choose to drive the system towards values other than the climatological mean and variance. The variance limitation can be viewed as a simple strategy to numerically stabilize the filtering process. As such, it may be more beneficial to choose values differing from climatology which are given by numerical necessities rather than by dynamical and physical considerations. For example, in non-equilibrium situations where the dynamics are locally off the attractor such as in the presence of strong fronts, the climatological values are not meaningful. Our approach still provides a means to control the dynamically natural overestimation to numerically stabilize the filtering procedure, however this would imply careful tuning of the limiters Aclim and aclim. Furthermore, it would be computationally impossible in a full-scale model to treat all of the unobserved variables as pseudo-observables, so one would need to choose carefully to which of the unobserved variables the VLKF should be applied.
It was noted in Gottwald et al. (2011) that the RMS errors are not sensitive to uncertainties in the estimation of aclim and Aclim, which may be obtained from historical data or from long time simulations of numerical models. However, instead of damping all pseudo-observables towards the same spatially homogeneous climatological value aclim, one may wish to damp towards spatially inhomogeneous finite-time statistics. One may, for example, use a running mean of the past history of analyses za instead of the constant aclim we have used here. However, for the long observation times used here, the analysis is not tracking (cf. Figure 4). Therefore driving the pseudo-observables towards non-climatological values of aclim will introduce on average larger RMS errors, degrading the skill. It is exactly the situation of large observational intervals where the quality of an analysis cannot be measured by how well it estimates the actual truth but by how well the statistics of the truth are reproduced. The situation of large observation intervals and sparse observational grids is particularly relevant for climate reanalysis in the pre-radiosonde era. In this case, long time climatology was shown to be beneficial as additional information in data assimilation. Our results show that model error may provide the necessary ensemble spread needed to combat finite ensemble size effects, similarly to covariance inflation; VLKF controls this spread by limiting unrealistically large analysis-error covariances.
The problem we addressed here is closely related to forecast bias-correction schemes (Takacs, 1996). Dee and Da Silva (1998) introduced a filtering strategy where a forecast bias is estimated using a running mean which was then subtracted from the forecast prior to the analysis. This strategy was applied to the Goddard Earth Observing System moisture analysis (Dee and Todling, 2000). We are dealing here with the additional problem of sparse observational grids. However, both approaches gain their skill improvement from driving the dynamics towards a mean which was estimated from the preceding dynamics using a running average rather than allowing the analysis to drift away due to forecast bias caused, as in our case, by model error.
We have used here underdamping as a model for numerically unstable dynamic cores; our forecast models still produce stable dynamics, albeit with significantly increased covariances. The effectiveness of our approach at controlling actual numerical instabilities remains an open question planned for further research. We mention the work by Grote and Majda (2006) who addressed a similar problem and designed a filter which assures an observability condition in the case of unstable dynamics in the dynamic core where the unstable dynamics are tamed by appropriately incorporating observations. Furthermore, we have not addressed the important issue raised in the introduction as to whether the proposed method is able to produce realistic energy spectra and control the energy dissipation rates. This also may require tuning of the limiters and is planned for further research.
We thank Sebastian Reich for bringing the problem of underdamping model error to our attention. LM acknowledges the support of an Australian Postgraduate Award. GAG acknowledges support from the Australian Research Council.