### Abstract

- Top of page
- Abstract
- 1. Introduction
- 2. ARMAX models and State Space
- 3. Wavelet transformation and autocorrelation shell representation
- 4. Model efficiency
- 5. Error correction of EPS
- 6. Conclusion
- Acknowledgements
- References

Within the EU Project PREVention, Information and Early Warning (PREVIEW), ensembles of discharge series have been generated for the Danube catchment by the use of various weather forecast products. Hydrological models applied for streamflow prediction often have simulation errors that degrade forecast quality and limit the operational usefulness of the forecasts. Therefore, error-correction methods have been tested for adjusting the ensemble traces using a transformation derived with simulated and observed flows. This article presents first results of the combination of state-space models and wavelet transformations in order to update errors between the simulated (forecasted) and the observed discharge. Copyright © 2008 Royal Meteorological Society

### 1. Introduction

- Top of page
- Abstract
- 1. Introduction
- 2. ARMAX models and State Space
- 3. Wavelet transformation and autocorrelation shell representation
- 4. Model efficiency
- 5. Error correction of EPS
- 6. Conclusion
- Acknowledgements
- References

The Joint Research Centre (JRC) is running the European Flood Alert System (EFAS) in preoperational mode in order to provide national water authorities with early warnings based on medium-range weather forecasting. The objective of the PREVIEW project is to examine the added value of an extended 3–10 days flood forewarning system, including probabilistic forecasts based on ensembles on the upper Danube catchment area (upstream Bratislava). LISFLOOD, a distributed, raster-based, combined rainfall–runoff and hydrodynamic model embedded in a dynamic geographic information system (GIS) environment (De Roo [1999] and De Roo *et al.* [2000]), is used at the JRC as a hydrological forecasting system. This flexible tool makes it possible to simulate hydrological processes and floods on a wide range of temporal and spatial scales taking all kinds of available meteorological forecast information as input. More details of the model structure and of the involved physical processes can be found in De Roo *et al.* [2002].

The purpose of forecasting floods is to minimize loss of life and injuries to people, loss and damage to property, and disruption of normal activities caused by flooding. While it is desirable to completely prevent all of these outcomes, this is not always possible, and the objective becomes that of reducing the flood magnitude and/or mitigate the negative effects of flooding. The desirable characteristics of a good flood forecast are: timeliness, accuracy and reliability.

The exact requirements under each heading will depend on the individual circumstances and are influenced by a large number of factors, which include the range of practical responses, the size and response time of the catchment and river system and the type of precipitation.

*Timeliness* The time between making a forecast of an event and its occurrence is called the lead time. Naturally, the longer the lead time is, the more are the opportunities for flood control or modification and for damage mitigation. If sufficient time is available and accurate predictions of the area to be affected are available, then evacuation even of relatively large numbers of people may be possible.

*Accuracy* Accuracy usually relates to the correctness of the forecasts of the magnitude and time of the flood peak and of the resulting water levels. In exceptional circumstances, it may relate to the forecasts of the complete hydrograph of the flood. The more accurate the forecast is, the better will be the implementation of flood control/modification and damage mitigation measures. These measures have a saving in cost, both financial and in physical and psychological injury.

*Reliability* The long-term reliability of the system can be assessed by its performance in two respects. It should always forecast a flood when it occurs and it should not forecast floods when it does not occur. The reliability, like accuracy, affects the confidence in deciding on response measures. Public perception of its reliability, or lack of reliability, may be very important in determining whether or not its warnings are heeded.

Since the objective of the EFAS is to provide end users with early warnings (lead times greater than 2 days) on European scale, one of the most important things is to estimate the time of exceedance of predefined thresholds as accurate as possible. Because rainfall–runoff models are far from being perfect, hydrologists doing operational forecasting need to put the model in better compliance with the current observations prior to using it in forecasting mode. This kind of error correction has also been termed ‘updating in hydrology’.

Recently Seo *et al.* [2006] presented a simple, parsimonious statistical procedure for the bias correction of an Ensemble Streamflow Prediction (ESP) system that accounts for hydrologic uncertainties in short range (1–5 days ahead), which is intended for operational use.

O'Connell and Clarke [1981] and Refsgaard [1997] reported on four different methodologies used for model updating. These methodologies depend on what is considered to be the main cause of discrepancy between observed and computed streamflow values. Therefore, one can distinguish between input updating, state updating and parameter updating (WMO [1992] and Madsen and Skotner [2005]). The most widely used updating procedures update the state variables or the output variables. The fourth procedure considers the actual streamflow as the sum of the model output and an error term. In this approach, the error term has to be modeled to allow for a prediction about its short-term realizations (error correction). Toth *et al.* [1999], for instance, used autoregressive moving average (ARMA) models to predict forecasting errors of a deterministic rainfall–runoff model, and Goswani *et al.* [2005] assessed the performance of eight real-time updating procedures, based mostly on error prediction. The advantage of this approach is that it can be easily applied to complex models such as full hydrodynamic flood propagation models (Madsen and Skotner [2005]).

In this article an autoregressive moving average with exogenous input (ARMAX) model in state-space form, also called dynamic linear model (DLM), is applied for updating. State-space models have been introduced in Kalman [1960] and Kalman and Bucy [1961], and an excellent treatment of ARMAX models and their equivalent state-space form is given by Shumway and Stoffer [2006]. The application of the DLM for the gauging station Hofkirchen (Bavaria) is shown in the next section. According to the fact that the errors between observed and simulated series occurs across many different time-scales and across different levels of resolution, the river flow discharge is decomposed by the use of wavelet transformations first. This decomposition, which will be shown in Section 3, can provide the detailed model error at different levels in order to estimate the state variables more precisely. Furthermore, some details about the application of wavelet transformation in forecast mode are given, and a possible way of solving the boundary problems is shown. Finally, this methodology of error correction combining state-space models and wavelet transformations, is applied to the Danube basin upstream Bratislava and some initial results are shown.

### 2. ARMAX models and State Space

- Top of page
- Abstract
- 1. Introduction
- 2. ARMAX models and State Space
- 3. Wavelet transformation and autocorrelation shell representation
- 4. Model efficiency
- 5. Error correction of EPS
- 6. Conclusion
- Acknowledgements
- References

The ARMAX model is a generalization of an ARMA model, which is capable of incorporating an external, (*X*), input variable. The linear time-invariant ARMAX representation is

- (1)

where *y*_{t} is a *p*-dimensional vector of observed output variables, *u*_{t} is an *m*-dimensional vector of input variables, *e*_{t} is a *p*-dimensional unobserved disturbance vector process, and *A*, *B*, and *C* are matrices of the appropriate dimension in the lag (back shift) operator L. Note that the time convention here implies that the input variable *u*_{t} can influence the output variable *y*_{t} in the same time period. This convention is not always used in time-series models, but is important for the application in error correction.

Literature on the ARMAX model and its generalization is rich (e.g. Brockwell and Davis [2002]). In the proposed methodology, the ARMAX model will be utilized to take the simulated discharge series (forecasts) into account, so the forecasting method of the error does not solely depend on historical (observed-simulated) data.

In the general case of ARMAX models, forecasts and their mean-square-prediction errors can be obtained by using the state-space formulation of the model and the Kalman filter.

A linear time-invariant statespace representation in innovations form is given by

- (2)

- (3)

where *z*_{t} is the unobserved underlying *n* dimensional state vector, *F* is the state transition matrix, *G*, the input matrix, *H*, the output matrix, and *K*, the Kalman gain.

The more general noninnovations form is given by

- (4)

- (5)

where *n*_{t} is the system noise, *Q*, the system noise matrix, and *R*, the output (measurement) noise matrix.

For an innovations form model, the state is defined as an expectation given past information, so the Kalman filter estimates the state exactly. For a noninnovations form model, the filter and smoother give slightly different estimates. An innovations form model would usually be specified based on some additional information about the structure of the system, typically a physical understanding of the system in engineering. In the absence of this, an arbitrary technique is to use a Cholesky decomposition to convert an innovations form model to a noninnovations form model.

For model estimation, the freeware R (available at http://cran.r-project.org) has been used. Especially, the package called DSE (developed by Gilbert [1995]), which focus on various aspects of multivariate ARMAX models, state-space modeling and Kalman filtering, has been applied.

In a first step, the state-space model has been estimated for a period of four years (from 1 October 1997 to 30 September 2001) of observed and simulated daily discharge data, taking the observed data as state vector and the simulated data as input vector. In this most simple version of the state-space model, the various matrices defining its dynamics are time-invariant, i.e. the model is constant. Starting from 1 October 2001, the model is run in simulation updating mode; that means that for each day the model is updated by taking the previous observed discharge value into account and predicting for the next 20 days, taking the simulated series as input for the model. In real time, the simulated series would represent the output of the hydrological model, taking the weather forecast as input. In case of having an ensemble of forecasts, the same procedure will be applied for each ensemble member. Since the methodology is tested in a hindcast mode, the simulated discharge data are computed by the use of observed precipitation and temperature data as input to the LISFLOOD model.

The next step is to decompose the discharge data by the use of wavelet transformation, and to apply the state-space model for each resolution level. The reason why this decomposition has been applied is that the difference between observed and simulated discharge will be caused by different time-scale processes.

In Figure 1, it can be seen, that the range of scales for the error that occurs at springtime (end of March 2002) and is caused by long lasting snowmelt processes, is by far larger than the error that appears in August 2002 and is caused by an overlapping of stratiform precipitation and convective rain fields of short duration.

The continuous wavelet transform of a continuous function produces a continuum of scales as output. More details about the applied continuous wavelet transform can be found in Carmona *et al.* [1998]. On the other hand, input data is usually discretely sampled and furthermore, a dyadic or twofold relationship between resolution scales is both practical and adequate. The latter two issues lead to the discrete transformation.

### 3. Wavelet transformation and autocorrelation shell representation

- Top of page
- Abstract
- 1. Introduction
- 2. ARMAX models and State Space
- 3. Wavelet transformation and autocorrelation shell representation
- 4. Model efficiency
- 5. Error correction of EPS
- 6. Conclusion
- Acknowledgements
- References

Wavelet decomposition provides a way of analyzing a signal in both time and frequency domains. They have been used effectively for image compression, noise removal, object detection, and large-scale structure analysis, among other applications. Especially, multiscale analysis of time series has appeared in the literature at an ever increasing rate. For a suitably chosen mother wavelet function ψ, a function *f* can be expanded as:

- (6)

where the functions Ψ(2^{j}*t* − *k*) are all orthogonal to each other. The coefficient *w*_{jk} gives information about the behavior of the function *f* concentrating on the effects of scale around 2^{−j} near time *t* × 2^{−j}. This wavelet decomposition of a function is closely related to a similar decomposition (the discrete wavelet transform, DWT) of a signal observed in discrete time.

In time-series analysis, DWT often suffers from a lack of translation invariance. This means that DWT-based statistical estimators are sensitive to the choice of origin. This problem can be tackled by means of a redundant or nondecimated wavelet transform. A redundant transform based on a *N*-length input time series has a *N*-length resolution scale for each of the resolution levels of interest. Therefore, information at each resolution scale is directly related at each time point. The maximal overlap DWT (MODWT), which is a special version of nondecimated DWT, has proven useful in analyzing various geophysical processes (Percival and Mofjeld [1997] and Serroukh *et al.* [2000]).

The Daubechies- and Morlet-wavelet transforms have been increasingly adopted by signal- and image-processing researchers. The Daubechies wavelets exhibit good trade-off between parsimony and information richness, while Morlet wavelets, on the other hand, have a more consistent response to similar events but have the weakness of generating many more inputs than the Daubechies wavelets for the prediction models. The main objective of the procedure will be the prediction of the next values, which points to the critical importance of the final values. The observed and simulated time series are finite of size, say *N*, and values at times N, N–1, N–2, …, are of greatest interest. Any symmetric wavelet function is problematic for the handling of such a boundary (or edge). It will not be possible to use wavelet coefficients, if these coefficients are calculated from unknown future data values. An asymmetric filter would be better for dealing with the edge of importance. Although one could hypothesize future data based on values in the immediate past, there is nevertheless, discrepancy in fit in the succession of scales, which grows with scale as larger numbers of immediately past values are taken into account. In addition, for both symmetric and asymmetric functions, a variant of the transform is needed, that handles the edge problem. Usually, this will be the mirror or periodic border handling. Although all these methods work well in a lot of applications, they are very problematic in forecast applications as they add artifacts in the most important part of the signal: its right border values. In order to show the relevance of this boundary problem for forecasting purposes, the following methods have been compared:

- 1.
Transformation of the whole series of observed and simulated data (using MODWT with Debauchies wavelets) at once into the wavelet domain and apply the state-space model directly on these transformed series for estimation (calibration) and for prediction in forecast mode. For each level of resolution, a state-space model has been fitted to the wavelet coefficients. The periods for the calibration (1 October 1997 to 30 September 2001) and for the prediction (1 October 2001 to 31 August 2002) were the same as for the simple state-space model without decomposition. Previous analysis of the wavelet covariance between the observed and simulated data (see Whitcher *et al.* [2000] for more details), have indicated that a maximum of five different levels is sufficient for these kind of data to capture the relevant error features. This method will be called MODWT-hindcast.

- 2.
The state-space models, estimated by method 1, will be used, but for the prediction period the models will be run in forecast mode, i.e. for each day, the next 20 days of simulation will be transformed quasi in real time. This method will be called MODWT-forecast.

- 3.
An autocorrelation shell methodology, which will be explained below, has been applied using an à trous algorithm (see Dutilleux [1987] and Shensa [1992]), where the boundary effects have been removed.

For example, when the MODWT is applied with periodic boundary conditions, the resulting wavelet and scaling coefficients are computed without making changes to the original series, which are treated as if they were circular. The boundary regions are dependent upon the resolution level and increase as the wavelet and scaling filter width increases. The filter width for the D4 (Daubechies orthogonal wavelets with 4 vanishing moments) can be computed by the general formula:

- (7)

where *L* = 4 for the D4 wavelets and *j* is the resolution level. In Figure 2, the boundary effect for the MODWT-hindcast and for the MODWT-forecast for the resolution level 2 and 3 is shown, where the edge of the affected boundary region is calculated according to (7). In that case, the MODWT-hindcast wavelet coefficients are representing the ‘true’ coefficients, since the transformation has been applied on the whole time series at once and the shown coefficients are from the beginning of the prediction period (starting from the 1 October 2001), whereas the MODWT-hindcast wavelet coefficients will be affected only close to the end of the prediction period. The coefficients inside the left and the right side boundary are identical, but one can clearly see the differences outside the boundaries. When the resolution decreases, i.e. the signal gets smoother and smoother, this effect increases and the boundary regions are also getting extended. According to (7), the decomposition of the data into five levels means that the last 46 coefficients are affected by future (unknown) values. As a consequence of this, the mirror or periodic border handling leads to unrealistic predictions.

To alleviate this boundary problem, a redundant representation using dilations and translations of the autocorrelation functions of compactly supported wavelets (the autocorrelation shell), will be used instead of the wavelets *per se* (Saito and Beylkin [1993]). An à trous algorithm is used to realize the shift-invariant wavelet transforms similar to the approach of Aussem *et al.* [1998]. The exact filters for the decomposition are the autocorrelation coefficients of the quadrature mirror filter coefficients of the compactly supported wavelets. The decomposition filters are, therefore, exactly symmetric. The recursive definition of the autocorrelation functions of compactly supported wavelets leads to fast recursive algorithms to generate the multiresolution representations. One of the interesting features of this representation is its convertibility to the orthonormal shell of the corresponding compactly supported wavelets on each scale, independently of other scales. The algorithm for such conversion is discussed in detail in Saito and Beylkin [1993].

By definition, the autocorrelation functions of a compactly supported scaling function ϕ(*x*) and the corresponding wavelet ψ(*x*) are as follows:

- (8)

- (9)

Since the function Φ(*x*) is exactly the autocorrelation function of compactly supported scaling functions, there is a one-to-one correspondence between the symmetric iterative interpolation scheme, introduced in Dubuc [1986] and Deslauriers and Dubuc [1989], and compactly supported wavelets (Ansari *et al.* [1991]). In general, the scaling function corresponding to Daubechies's wavelet with *M* vanishing moments leads to an iterative interpolation scheme which uses the Lagrange polynomial of degree 2*M* − 1 (Deslauriers and Dubuc [1989]).

Since the filter coefficients *p*_{k} are obtained by evaluating the Lagrange polynomials at the origin *x* = 0, the filter coefficients for the edges can be adjusted by simply generating them by evaluating these polynomials at the desired points (Beylkin and Saito [1997]). In this way, the boundary problem can be solved. Using the filters *P* and *Q*, the pyramid algorithm for expanding into the autocorrelation shell can be obtained as:

- (14)

- (15)

These shell coefficients obtained from (15) can be used to directly reconstruct the signals. Given the smoothed signal at two consecutive resolution levels, the detailed signal can be derived as:

- (16)

In the following section, this method of wavelet decomposition by autocorrelation shell representation in combination with state-space models (WSS model) will be applied and evaluated.

### 4. Model efficiency

- Top of page
- Abstract
- 1. Introduction
- 2. ARMAX models and State Space
- 3. Wavelet transformation and autocorrelation shell representation
- 4. Model efficiency
- 5. Error correction of EPS
- 6. Conclusion
- Acknowledgements
- References

In Figure 3, the PME and Nash-Sutcliff efficiency are shown for three different models running in forecast mode for the hydrological year 2002: (1) WSS-operational, which corresponds to method 3 in the previous section, (2) State-space model (ARMAX), without wavelet transformation, and (3) WSS-hindcast, which corresponds to method 1 in the previous section.

The newly developed WWS-operational model shows some clear improvements in comparison to classical ARMAX-based error-correction methods, especially, at lead times greater than 2 days, which is relevant for the EFAS. It is interesting to see how the boundary effect (Section 2) gets diminished after lead times of 13 days for the WSS-hindcast model. Up to the lead time of 12 days the efficiency of the WSS-hindcast model is superior, simply because of the fact, that the wavelet coefficients close to boundary are affected by future values, that are only known in running the model in hindcast mode and will not be known in the forecast model. The results of running the model in forecast mode by the use of periodic boundary conditions are not shown, because the efficiency for that case is even worse than the efficiency of the uncorrected model simulation.

### 5. Error correction of EPS

- Top of page
- Abstract
- 1. Introduction
- 2. ARMAX models and State Space
- 3. Wavelet transformation and autocorrelation shell representation
- 4. Model efficiency
- 5. Error correction of EPS
- 6. Conclusion
- Acknowledgements
- References

The next step is to apply this error-correction method to ensembles of discharge data generated by the use of meteorological EPS as input to the LISFLOOD model. In the observation period of the hydrological year 2002, there have been two major flood events. For the first one in March 2002, which was caused by snowmelt and rainfall, only the EPS of ECMWF have been available, whereas for the second event in August 2002 also, the higher-resolution ensembles of COSMO-LEPS (with spatial resolution of ∼10 km and a forecast horizon of 5.5 days) from ARPA-SIM and the VAREPS (with ∼40 km resolution for forecast day 1–7 and with ∼80 km resolution for forecast day 8–14) from ECMWF have been made available.

The results for the August 2002 flood are shown in Figure 4 and Figure 5, initiated 9 and 7 days, respectively, before the observed flood peak. The timing accuracy, which is one of the most important objective for an early warning system, has been improved by the use of this error-correction method, especially for the COSMO-LEPS, which can be seen in Figure 4 (b). Besides this, the spread of the ensemble traces has been reduced significantly. Whereas the forecast based on VAREPS starts to show the flood 2 days after the first occurrence of the possible flood predicted by COSMO-LEPS (Figure 5 (a)), at this time the flood is predicted almost perfectly already 6 days in advance, and at that forecasting horizon the error-correction method also starts to improve the system.