### Abstract

- Top of page
- Abstract
- 1. Introduction
- 2. Estimating the Probability Distribution of the Forecast Error
- 3. Case Studies
- 4. Results
- 5. Concluding Remarks
- Acknowledgments
- References
- Supporting Information

[1] A method for quantifying the uncertainty of hydrological forecasts is proposed. This approach requires the identification and calibration of a statistical model for the forecast error. Accordingly, the probability distribution of the error itself is inferred through a multiple regression, depending on selected explanatory variables. These may include the current forecast issued by the hydrological model, the past forecast error, and the past rainfall. The final goal is to indirectly relate the forecast error to the sources of uncertainty in the forecasting procedure, through a probabilistic link with the explaining variables identified above. Statistical testing for the proposed approach is discussed in detail. An extensive application to a synthetic database is presented, along with a first real-world implementation that refers to a real-time flood forecasting system that is currently under development. The results indicate that the uncertainty estimates represent well the statistics of the actual forecast errors for the examined events.

### 1. Introduction

- Top of page
- Abstract
- 1. Introduction
- 2. Estimating the Probability Distribution of the Forecast Error
- 3. Case Studies
- 4. Results
- 5. Concluding Remarks
- Acknowledgments
- References
- Supporting Information

[2] The development and improvement of real-time flood forecasting systems is a very topical subject today. In fact, the effectiveness of early warning systems for the sake of reducing the damages and casualties induced by floods is widely recognized and new avenues of research are continuously open by the recent technical advances in computer sciences, hydrological monitoring and modeling. Among the most crucial scientific interests the assessment of the forecast uncertainty plays a not negligible role. In general, a forecast does not eliminate uncertainty about a future event, but only reduces it. Therefore it is extremely important to quantify the residual uncertainty, in order to make the forecast useful for practical purposes [*Krzysztofowicz*, 2001]. A timely communication to end users requires though that the estimate of the uncertainty is performed quickly.

[3] The need for an effective quantification of the forecast uncertainty has been recently stressed by *Krzysztofowicz* [2001, 2002], who remarked that the prevailing format of operational hydrological forecasts is still deterministic. Information neglecting uncertainty may induce a misleading illusion of certainty in the end user or decision maker. In fact, when such a forecast turned out to be wrong, the consequences would probably be worse with respect to a situation where no forecast was available.

[4] Uncertainty assessment is the subject of an intense research activity in hydrology [*Beven*, 2006a, 2006b; *Montanari*, 2007; *Mantovan and Todini*, 2006]. Basically, the techniques so far proposed for tackling the problem can be separated into statistical and non statistical approaches. While many researchers are convinced that all the hydrological forecasts should be expressed in terms of probabilities [*Krzysztofowicz*, 2001], others are convinced that ergodicity and stationarity, which are the basis for statistical inference, cannot be suitable working hypotheses in hydrology. The main reason is that the underlying physical processes are highly heterogeneous in space and time and therefore might be inherently non stationary. From there comes the motivation for using non statistical approaches to assess uncertainty in hydrology, like, for instance, the generalized likelihood uncertainty estimation (GLUE) [*Beven*, 2006a].

[5] No matter which method is applied, uncertainty assessment in hydrological forecasting is still considered a relevant practical problem. A valuable contribution was recently provided by *Krzysztofowicz* [2001, 2002] who developed the Bayesian forecasting system (BFS), a statistically based method for assessing the uncertainty of flood forecasts. This technique presents many advantages. The most relevant one is the capability to take explicitly into account the uncertainty of the precipitation forecast, if this latter is expressed in the form of a probability distribution of future rainfall.

[6] The present study aims to propose a statistical method for quantifying the total uncertainty in hydrological forecasting, which is based on the use of a meta-Gaussian statistical model to infer the probability distribution of the forecast error depending on selected explanatory variables. The meta-Gaussian multivariate probability distribution was introduced in hydrology by *Kelly and Krzysztofowicz* [1997]. It is obtained by fitting with a multivariate Gaussian distribution random variables with arbitrary marginal distributions by embedding the normal quantile transform (NQT) of each variate into the Gaussian law. For more details see section 2.

[7] The proposed method does not attempt to separately estimate the contribution of each individual source of uncertainty. It is operationally simple and fast and relies on mild assumptions that are frequently met in practical applications. In view of such features, this technique might provide useful perspectives for estimating the uncertainty of hydrological forecasts in real time within a robust framework.

### 2. Estimating the Probability Distribution of the Forecast Error

- Top of page
- Abstract
- 1. Introduction
- 2. Estimating the Probability Distribution of the Forecast Error
- 3. Case Studies
- 4. Results
- 5. Concluding Remarks
- Acknowledgments
- References
- Supporting Information

[8] In order to estimate the uncertainty of hydrological forecasts, it is assumed here that the forecast error is a stationary and ergodic stochastic process, denoted with the symbol *E*(*t*). It is suggested that its statistical properties are inferred by analyzing a past realization *e*_{obs}(*t*) = *Q*_{obs}(*t*) − *Q*_{pred}(*t*) that it is assumed to be available, where *Q*_{obs}(*t*) and *Q*_{pred}(*t*) are true and forecasted river flows, respectively. Therefore, this method implies that the hydrological model is preliminarily applied in order to predict past observations by emulating an operational forecasting framework. In this way, the past realization of the forecast error, *e*_{obs}(*t*), can be obtained.

[9] In order to derive a probabilistic model for *E*(*t*), the main statistical behavior of the forecast error have to be taken into account. They can be summarized in the following two points. (1) *E*(*t*) is characterized by marginal statistics that change in time. Typically, the greater the predicted hydrological variable, the greater the forecast error. (2) *E*(*t*) is frequently affected by a strong persistence. However, such persistence does not mean that the error can be easily predicted. Generally, a significant forecast error is an indicator of the presence of relevant uncertainty in the predicting procedure at the forecast time and thus it is likely to be followed by high errors as well. However, the realizations *e*_{obs}(*t*) of hydrological forecast errors are usually characterized by infrequent and random changes of sign. These induce the presence of a sizable uncertainty in the prediction of the next forecast error.

[10] The use of a meta-Gaussian model is then proposed to derive the time varying probability distribution of the forecast error. Basically, the probability distribution of *E*(*t*) is inferred on the basis of its dependence on *M* selected explanatory random variables. These are in charge of explaining the variability in time of the marginal statistics of *E*(*t*). The statistical inference is performed in the Gaussian domain, by preliminarily transforming *E*(*t*) and the explanatory variables to the Gaussian probability distribution. The above transformation is operated through the Normal Quantile Transform (NQT).

[11] Let us refer to *E*(*t*) for explaining the NQT, which involves the following steps: (1) for the *j*th data *e*_{obs}(*t*_{j}) of the realization *e*_{obs}(*t*) the cumulative frequency *F*[*e*_{obs}(*t*_{j})] is computed by using the Weibull plotting position [*Stedinger et al.*, 1993], that is, *F*[*e*_{obs}(*t*_{j})] = *k*_{j}/(*n* + 1). Here *k*_{j} is the position occupied by *e*_{obs}(*t*_{j}) in the sample sorted in ascending order and *n* is the sample size of *e*_{obs}(*t*). (2) For each *F*[*e*_{obs}(*t*_{j})] the standard normal quantile *Ne*_{obs}(*t*_{j}) is computed and associated with the corresponding *e*_{obs}(*t*_{j}). Thus, a discrete mapping from *e*_{obs}(*t*) to its transformed counterpart *Ne*_{obs}(*t*), which gives the NQT, is obtained. In order to be able to apply the inverse of the NQT, that is NQT^{−1}, for any value of the transformed forecast error, a linear interpolation is used to connect the points of the discrete mapping previously obtained. The region beyond the minimum and maximum of *Ne*_{obs}(*t*) is covered by linear extrapolation. For more details about the operational use of the NQT see *Kelly and Krzysztofowicz* [1997] and also *Montanari and Brath* [2004].

[12] In practice, the probabilistic model for *E*(*t*) is built as follows. First of all, it is assumed that positive and negative errors come from 2 different statistical populations *E*^{(+)}(*t*) and *E*^{(−)}(*t*). Therefore, the probability model for *E*(*t*) is given by a mixture of two probability distributions, one for *E*^{(+)}(*t*) and one for *E*^{(−)}(*t*). The mixture is composed such that the area of the probability distribution of *E*^{(+)}(*t*) is equal to the percentage, *P*^{(+)}, of positive errors over the total sample size of the available past realization *e*_{obs}(*t*) of the forecast error.

[13] The two realizations *e*^{(+)}_{obs}(*t*) and *e*^{(−)}_{obs}(*t*) are transformed through the NQT, therefore obtaining the normalized realizations *Ne*^{(+)}_{obs}(*t*) and *Ne*^{(−)}_{obs}(*t*). Then, *M* explanatory variables, *X*^{(i)}(*t*) with *i* = 1,…, *M* (which should be readily available at the forecast time) are selected in order to explain the variability in time of the marginal statistics of *E*^{(+)}(*t*) and *E*^{(−)}(*t*). The values of such explanatory variables for the realizations *e*^{(+)}_{obs}(*t*) and *e*^{(−)}_{obs}(*t*) above are estimated and then transformed by using the NQT, therefore obtaining the normalized explanatory variables *Nx*^{(i)}_{obs} (*t*) with *i* = 1, …, *M*.

[14] In the Gaussian domain, it is assumed that the forecast error can be expressed as a linear combination of the selected explanatory variables. Let us focus on the positive error. The linear combination can be expressed through the following relationship:

where ɛ^{(+)}(*t*_{j}) is an outcome of a homoscedastic and Gaussian random variable. An analogous relationship holds for *Ne*^{(−)}(*t*). It is assumed that positive and negative errors are conditioned by the same explanatory variables, but the fit of the linear regression (1) leads to a different set of coefficient values. Such coefficients are estimated by plugging in (1) the past realizations of transformed forecast error, *Ne*^{(+)}_{obs}(*t*), and explanatory variables, *Nx*^{(i)}_{obs} (*t*), and then by identifying the coefficient values that lead to the best fit (for instance by minimizing the sum of the squares of ɛ^{(+)}(*t*_{j})).

[15] The goodness of the fit provided by (1) can be verified by drawing a normal probability plot and a residual plot for ɛ^{(+)}(*t*) as in the work by *Montanari and Brath* [2004]. In the case the goodness-of-fit test is not satisfied, a better result can be obtained by calibrating the regression on the basis of the data points corresponding to the higher river flows only. Goodness-of-fit checking for the applications reported here is shown in section 4.1.

[16] Once the linear regression (1) has been calibrated, for positive as well as for negative errors, the probability distribution of the transformed positive forecast error can be easily derived for potential real-time and real-world applications. Such distribution is Gaussian and is expressed by the following relationship:

where ∼ means equality in probability distribution and *G* indicates the Gaussian distribution whose parameters are given by

Analogous relationships (from (2) to (4)) hold for the negative error. Therefore, the confidence bands (CBs) for the transformed forecast at an assigned significance level can be straightforwardly derived. In detail, the difference, (*t*_{j}), between the forecast and the upper CB, in the Gaussian domain, at the *α* significance level is given by the 1 − *α*/(2·*P*^{(+)}) quantile of the Gaussian distribution given by (2), (3) and (4). Given that *P*^{(+)} can be arbitrarily close to 0, in the technical computation one may obtain values greater than 1 of *α*/(2·*P*^{(+)}). This means that the probability of getting a positive forecast error is small enough to make equal to 0 the width of the upper CB at the *α* significance level.

[17] For instance, if *P*^{(+)} = 0.5 and *α* = 10%, (*t*_{j}) is given by the well known relationship

Finally, by applying back the NQT one obtains the CBs for the assigned significance level in the untransformed domain. It is important to put in evidence that the *α* significance level corresponds to the 1 − *α* confidence level. This means that the identified CBs of the hydrological forecast are such that there is a probability of 1 − *α* for the true value of the hydrological variable to fall between them.

[18] The reason why positive and negative errors are treated separately is that a good fit was not achieved through the linear regression (1) when the errors were pooled together. In fact, in this case, it appears that the NQT is not effective in making the errors homoscedastic and therefore the assumption of linearity does not hold. The reason for this result is that the NQT is not efficient in assuring homoscedasticity if the mean of the model error is not significantly changing across the range of the error itself, as it often happens when dealing with hydrological models. By treating positive and negative errors separately the problem disappears and the assumptions of the linear regression are met. Finally, it is important to note that the only assumption made about the sign of the future forecast error is that it has a probability equal to *P*^{(+)} to be positive. Therefore, no inference is made on the sign of the forecast error on the basis of the explanatory variables.

### 5. Concluding Remarks

- Top of page
- Abstract
- 1. Introduction
- 2. Estimating the Probability Distribution of the Forecast Error
- 3. Case Studies
- 4. Results
- 5. Concluding Remarks
- Acknowledgments
- References
- Supporting Information

[51] A statistical approach for assessing the total uncertainty of hydrological forecasts is proposed, which is based on the use of a probabilistic model of the forecast error. This is built by using a meta-Gaussian approach to infer the probability distribution of the forecast error on the basis of its dependence on selected explanatory variables. The procedure herein proposed was first tested by using an extensive synthetic data set. Subsequently, a second test was performed by using flood events observed on the Toce River basin, that were forecasted by a flood forecasting system. The synthetic case study is motivated by the need for an extended database in order to be able to perform an extensive test of the reliability of the method. The real-world case study is instead performed in order to prove that the method works well even with real data, which are not as well behaved as synthetic records.

[52] In both case studies the uncertainty assessment was satisfactory, even though both applications highlighted the relevant role played by the calibration data set. Rainfall-runoff models are often characterized by a significant variability of their performances and therefore it is important to calibrate the uncertainty assessment method on the basis of an extended and representative data set, in order to keep track of a comprehensive sample of past events and different hydrological conditions. As a matter of fact, a sizable variability of the rainfall-runoff model performances may reduce the reliability of the uncertainty assessment, especially if a short data sample is used for calibrating the method. One should note that the above requirement for an extended data set might be difficult to meet, as extreme events are by definition rare. This is a frequent problem of statistical methods for uncertainty assessment in hydrology, which is often emphasized by scientists who prefer to use non statistical approaches. The aim of this work is not to contribute to this specific debate but only to propose a tool and discuss its potential limitations.

[53] Other limitations of the proposed technique are the need to recalibrate the coefficients of the regression if the lead time of the forecast changes and the incapability to explicitly take into account the uncertainty in the precipitation forecast, which is accounted for implicitly. However, the technique presented here might be applied to the situation in which precipitation ensemble forecasting is available. In such a case, a ensemble confidence bands can be built for the river flow forecasting in the form of a mixture of the forecast probability distributions of each member of the ensemble. Finally, we would like to remark that the computational requirements and times of the proposed technique are extremely limited and therefore it can be successfully applied in real-time forecasting.