Separately accounting for uncertainties in rainfall and runoff: Calibration of event-based conceptual hydrological models in small urban catchments using Bayesian method



[1] Uncertainty analysis of hydrological models is usually based on model calibration, and the Bayesian method is a popular way to evaluate the uncertainty. The traditional Bayesian method usually uses lumped model residuals to form the likelihood function, where uncertainty in inputs (rainfall) is not explicitly addressed. This paper compares three approaches based on Bayesian inferences, considering rainfall uncertainty either implicitly or explicitly in calibration. Consistent parameter estimation and reliable quantification of predictive uncertainty are mainly examined. When rainfall uncertainty is explicitly treated in calibration, several rainfall observations at one-minute time steps are grouped to share one multiplier to consider the possible observation errors. The appropriate grouping strategy that balances the representativeness and the complexity of the problem is suggested. The application of the methods considered in this study focuses on small urban catchments (<200 ha) with a small temporal scale (1 min time step), in contrast to most literature studies dealing with larger catchments monitored at larger time steps. It is found that uncertainty in rainfall has a minor contribution to the total uncertainty in runoff estimation, and this minor role can be explained by the low pass filter effect of the linear reservoir model. However, the approach explicitly accounting for input uncertainty results in more informed knowledge for uncertainties related with hydrological model calibrations, which can possibly provide an estimation of uncertainty attributed to rainfall records. It should be noted that rainfall error estimates can compensate model structural uncertainty that is not explicitly addressed in this study.

1. Introduction

[2] Conceptual rainfall runoff models are widely used in various decision-making problems ranging from online flood forecasting to evaluation of flood-reducing strategies and hydraulic structure designs. Many parameters of such models cannot, in general, be obtained directly from measurable quantities, and hence model calibration is needed. A model is calibrated to determine values of model parameters so that the model simulates the hydrological behavior of a catchment as closely to observations as possible. Hence, model calibration is an essential step before a model can be confidently used. The confidence that can be ascribed to model simulations largely depends on how a model has been calibrated.

[3] The assessment of parameter and predictive uncertainty of hydrological models is an essential part of any hydrologic study [Schoups and Vrugt, 2010], and uncertainty analysis in hydrological models has attracted much attention. Uncertainty analysis in hydrological models is commonly carried out based on calibration. Model parameter uncertainties are generally estimated based on residual errors. The residual errors are summarized using a likelihood function to quantify the probability density of observed data being generated by a particular parameter set. The mapping from parameter space to likelihood space results in a range of plausible parameter sets and allows estimation of parameter and predictive uncertainty. The likelihood function can be specified through either a formal or an informal approach. An informal likelihood function is specified flexibly without strictly defined statistical assumptions on residual errors and the function is formed subjectively. A well-known example is the generalized likelihood uncertainty estimation (GLUE) methodology [Beven and Binley, 1992]. With a formal approach, the residual errors are assumed to obey certain statistical models using Bayesian statistics. The simpler model assumes that the errors are independent and identically distributed normal random variables with a mean of zero and a constant variance. Discussions and comparisons of informal and formal approaches are given by many researchers [e.g., Makowski et al., 2002; Montovan and Todini, 2006; McMillan and Clark, 2009; Jin et al., 2010; Dotto et al., 2012]. Among many possible methods for searching for possible model parameters, the Markov Chain Monte Carlo (MCMC) method has become increasingly popular, being applied in calibration for problems involving complex inference, search, and optimization [Vrugt et al., 2003]. The MCMC method is mostly coupled with Bayesian statistics using a formal likelihood function [e.g., Schoups and Vrugt, 2010], but it can also be applied using informal likelihood functions [e.g., Blasone et al., 2008].

[4] The above approaches calibrate models based on residual errors that typically consist of a combination of input, model structure, output, and parameter errors. Traditionally, uncertainty in parameters is mainly attributed to output errors in calibration, and the errors in inputs are not explicitly addressed. It is not realistic for real-world applications because measured forcing data (rainfall measurements in hydrological process in this case) often contain significant uncertainty [Kavetski et al. 2006a, 2006b; Stransky et al., 2007; Vrugt et al., 2009], and hence it is desirable to incorporate and treat different uncertainty sources appropriately. Kavetski et al. [2006a, 2006b] and Vrugt et al. [2008] made the step forward to incorporate input uncertainty in hydrological model calibration by introducing multipliers to rainfall events. Kavetski et al. [2006a, 2006b] developed the Bayesian total error analysis (BATEA) framework to explicitly represent each source of uncertainty in hydrological models. BATEA relies on a hierarchical Bayesian model to handle uncertainty in terms (e.g., rainfall) by using latent variables that will be identified in the calibration as other model parameters. In addition, the framework also allows modelers to introduce as much detailed information about the data accuracy and uncertainty description as is known by simply incorporating parameters in error models into the hierarchical Bayesian model. In comparison to traditional approaches that lump all uncertainties into a single error term, BATEA provides improvements on model calibration in terms of deriving consistent parameter estimation and reliable quantification of predictive uncertainty [e.g., Kuczera et al., 2006, 2010; Thyer et al., 2009; Renard et al., 2010, 2011; Salamon and Feyen, 2010; Li et al., 2012]. Explicit input uncertainty analysis under the Bayesian context using the BATEA framework has been applied to different extents [e.g., Huard and Mailhot, 2006, 2008; Reichert and Mieleitner, 2009; Sikorska et al., 2012]. Vrugt et al. [2008] also evaluated rainfall uncertainty simultaneously with the hydrologic model parameters by minimizing the mismatch between observed and simulated catchment responses. Vrugt et al. [2008] did not directly include random effects of rainfall errors as a multiplicative factor in the likelihood term, which differs from the BATEA.

[5] This paper addresses uncertainty estimation in hydrological model calibration with a focus on handling rainfall uncertainty. Three approaches based on the Bayesian concept, considering input uncertainty in hydrological model calibration in various ways, are presented and compared. This paper focuses on the calibration of small urban catchments (<200 ha with high imperviousness of surfaces) with a small temporal scale (1 -min time step), while most literature studies model larger catchments at hourly or, more frequently, daily time steps. The paper investigates the ability of different approaches for providing parameter estimation and uncertainty quantification. The merits of explicit separation of input and output uncertainties are discussed. In addition, when explicitly incorporating rainfall errors in calibration, rainfall measurements at 1 min time step are grouped to share parameters to be calibrated to keep down the dimension of the calibration problem. The proper grouping strategies are studied. The remainder of the paper is organized as follows: section 2 presents the general hydrological calibration problem and the three approaches that account for input uncertainty in different ways; in section 3, application cases are introduced; section 4 then gives the results and discussion, focusing on comparison between different approaches and explanations of observed results; lastly conclusions are drawn.

2. Methodology

2.1. Conceptual Hydrological Model

[6] In this study, the hydrological process is described by a conceptual rainfall runoff model. This model consists of a relatively simple rainfall loss model and a routing model using two linear reservoirs. Such a model structure is frequently applied in urban hydrology for small impervious catchments drained by artificial sewer systems, where the contribution of natural soils are of negligible importance (except for very exceptional rain events) and where evaporation and evapotranspiration are also negligible at the rain event time scale. The rainfall loss model comprises two parts, i.e., an initial loss and a proportional loss during the wet period. Therefore, the net rain is calculated by:

display math(1)

where Inet is the net rainfall intensity (mm/h), I is the raw rainfall intensity (mm/h), Lini is the initial loss (mm), Pcons is the constant proportional rainfall loss. Inet is then converted to the inflow of the first reservoir with a lag time Tlag by multiplying with the catchment area A:

display math(2)

[7] The outflow Qout from the reservoir varies linearly with the storage volume V:

display math(3)

where the parameter K (min) is called reservoir constant. Combined with an equation of conservation of mass:

display math(4)

the analytical solution of equation (4) can be derived by integration of the differential equation over time interval [ inline image]:

display math(5)

[8] A second linear reservoir is placed in series after the first reservoir in the conceptual model to simulate the runoff. The reservoir constants of the two cascaded reservoirs are set the same because they are heavily correlated when assessing them in calibration according to some preliminary tests. A base flow q due to dry weather flow or ground water interactions is also considered as a part of contribution to runoff. The base flow q is a constant value simply added to the outflow from the second reservoir.

[9] The conceptual rainfall runoff model has five parameters needing to be calibrated, i.e., rainfall initial loss Lini, rainfall constant proportional loss Pcons, lag time of inflow Tlag, reservoir constant of the two reservoirs K, and base flow q.

2.2. Calibration for Hydrological Model: Estimate Uncertainty in Model Parameters

[10] Assuming that the structure of the hydrological model is predetermined and fixed, the model f can be written as

display math(6)

where y = {y1,…yn} is the observed values of outputs with length n, x = {x1,…xm} is the inputs with length m, ϴ = (ϴ1, …,ϴd) is a vector consisting of d model parameters, and e = {e1,…en} represents the residuals. The lengths of inputs and outputs are not necessarily the same due to different time steps being used and the existing catchment response time. Calibration is a reverse process that searches for model parameters when several sets of inputs and outputs are known. The classical approach for calibration endeavors to find the best values of parameters ϴ that make the vector of error terms e(ϴ) in some sense as close to zero as possible. The most common measure in parameter estimation is the sum of squared errors (SSE). However, the minimization of SSE only provides an estimate of the best values of ϴ. When uncertainty needs to be considered, the probability density function (PDF) of ϴ is desirable to help assess the information content of data and generate confidence intervals of predictive outputs [Vrugt et al., 2008].

[11] One approach to estimate uncertainty of parameters in model calibration is through Bayesian statistics. The Bayesian statistics considers model parameters ϴ as probabilistic variables having a joint posterior PDF, which captures the probabilistic belief about parameters ϴ according to the observed inputs and outputs. Bayesian theorem provides a way to update prior PDF to posterior PDF with observations:

display math(7)

[12] Let the simulated response be inline image. If the residual errors e = {e1,…,en} are mutually independent and Gaussian-distributed with a zero mean and a limited variance inline image, the posterior PDF has the following form:

display math(8)

where c is a normalizing constant and p(ϴ) signifies the prior distribution of ϴ. This posterior distribution combines the data likelihood (multiplicative term of equation (8)) with a prior distribution. Earlier studies using Bayesian concept frequently employed equation (8) to estimate uncertainty in model parameters.

2.3. Input Uncertainty Consideration

[13] Traditional parameter uncertainty estimation in model calibration using a formal likelihood approach does not take into account input uncertainty explicitly. The likelihood of different parameter sets is computed based on output residuals (see equation (8)). However, the validity of this approach is questioned when confronted with significant errors and uncertainty in model inputs [Vrugt et al., 2009; Deletic et al., 2012]. Different error sources need to be explicitly considered to be able to advance the field of hydrology and to help draw appropriate conclusions about parameter and model predictive uncertainty.

[14] Vrugt et al. [2009] used a single rainfall multiplier for each entire storm event to represent uncertainty in rainfall in a daily rainfall runoff model. They stated that by allowing rainfall multipliers to vary in hydrologically reasonable ranges, parameter inference and stream flow predictions are improved. In this case, the posterior distribution can be expressed as:

display math(9)

where inline image is the corrected rainfall.

[15] Still using Bayesian concept, both errors in inputs and outputs are viewed as independent variables that are Gaussian distributed with known variances; accordingly, the joint posterior PDF of parameter sets and corrected inputs (will be inferred in calibration as model parameters) can be written as:

display math(10)

[16] Equation (10) is consistent with the framework of BATEA [Kavetski et al., 2006a]. The explicit inclusion of input uncertainty in the posterior distribution makes the separation of input and output uncertainties possible, and it makes the approach theoretically sounder. In comparison to equation (9), equation (10) considers input errors as normally distributed variables, while in equation (9) input errors contribute only through model outputs. Consequently, when calibration is performed, the input errors are allowed to be varied less freely by equation (10) due to the direct inclusion of their effects as random variables into the posterior distribution expression.

[17] The logarithmic formulation of equations (8) to (10) is used to assure the computational stability in the MCMC algorithm. inline image is modeled with a multiplier ki following previous studies [e.g., Kavetski et al., 2006b; Vrugt et al., 2009]:

display math(11)

[18] Rainfall estimates are forced to zero in case of negative values. The standard deviations σx and σy can be evaluated according to measuring techniques and methods. It is worth mentioning that the likelihood functions in equation (10) are not restricted to the independent normal distributions as in the equation but should be determined according to prior knowledge on uncertainty. If parameters in error models (e.g., inline image and inline image) are not known, theoretically, their posterior distributions can also be inferred by adding them into the inference list. However, the inference of parameters in error models should be carefully implemented because vague information on uncertainty might lead to an ill posed problem when supplied with not sufficient prior knowledge of data uncertainty. Renard et al. [2010] recommended that the use of more precise statistical descriptions of errors reflecting actual knowledge makes the posterior distribution well posed.

[19] In principle, each rainfall observation error (at 1 min time step in urban catchments) can be viewed as an independent variable. However, this makes the dimensionality of the calibration problem grow massively and it might also be susceptible to overparameterization, thus to deteriorate the forecasting capability of the hydrological model [Vrugt et al., 2009]. The dimension of the calibration problem is reduced by grouping a number of rainfall observations at 1 min time steps, with each group having one single multiplier to consider uncertainty and to be calibrated, that is, in equation (11) values of ki are assigned a unique value for all successive xi in each group. Hence, the dimension of the calibration problem varies with the chosen number of rainfall observations in a group. When simulating one event, if all rainfall observations in this event are considered in one group, the calculation replicates the approach used by Kavetski et al. [2002, 2006a, 2006b] and Vrugt et al. [2009] applying one unique multiplier for a whole event. However, it should be noted that the time steps (1 min) in this study and in the mentioned literature studies (1 day) are quite different. The influence of the problem dimensions on uncertainty results will also be discussed in this study.

[20] This paper focuses on the treatment of rainfall uncertainty applied to small urban catchments at 1 min time steps. The model structural uncertainty which is generally present in real cases is not explicitly addressed because an entirely satisfactory method for handling model structural uncertainty is still lacking [Snowling and Kramer, 2001; Refsgaard et al., 2006; Renard et al., 2010]. The ignored model error might be compensated by rainfall error estimations. Considering this, the estimated rainfall errors for a specific model are clearly model dependent and should not be interpreted independently of the specified model.

2.4. MCMC Method and DREAM Scheme

[21] For most practical hydrological problems, the posterior PDF cannot be obtained analytically. Therefore, iterative approximation methods such as MCMC simulation have been developed. MCMC is a stochastic simulation that successively visits solutions in the parameter space with stable frequencies stemming from a fixed probability distribution. One of the mostly used MCMC methods is the Metropolis-Hasting algorithm that samples parameters using a proposal distribution, and its effectiveness is well established in hydrological modeling [e.g., Bates and Campbell, 2001; Dotto et al., 2011]. A typical MCMC sampler proceeds in three steps: First, a trial move from the current position of the Markov chain is generated; the newly generated candidate point is either accepted or rejected according to some rules (e.g., Metropolis acceptance probability); finally the chain moves to the new position if the proposal is accepted, otherwise the chain remains at the current location.

[22] The method of DREAM (differential evolution adaptive metropolis) based on MCMC was especially designed to efficiently estimate the posterior PDF of hydrologic model parameters [Vrugt et al., 2008]. DREAM uses an adapted MCMC method that has multiple chains in parallel to provide a robust exploration. DREAM has been applied and tested on various complex and high-dimensional hydrological problems [e.g., Schoups et al., 2010; Keating et al., 2010]. The DREAM toolbox coded in Matlab [Vrugt et al., 2008, 2009] is used and adapted for the three calibration approaches in this study.

2.5. Three Approaches

[23] Three approaches that consider input (rainfall) uncertainty in different ways are tested and compared in this study. Consistent parameter estimation and reliable quantification of predictive runoff are mainly examined. The first approach using equation (8) as the posterior distribution expression does not take input uncertainty into account explicitly, and the number of calibrated parameters is five (the number of model parameters). The second approach using equation (9) and the third approach using equation (10) incorporate input uncertainty by rainfall multipliers of a certain number of grouped observations, and the number of calibrated parameters is the sum of the number of model parameters and the number of multipliers. Table 1 lists the main characteristics of the three approaches.

Table 1. Main Characteristics of Three Approaches for Hydrological Model Calibration
ApproachesPosterior DistributionDescription
Approach 1Equation (8)It does not consider input errors explicitly.
Approach 2Equation (9)It incorporates input errors by rainfall multipliers. The influence of input errors is included in the posterior distribution expression through the model outputs as a function of inputs.
Approach 3Equation (10)It incorporates input errors by rainfall multipliers. The effect of rainfall errors is directly expressed in the posterior distribution expression with distribution assumptions.

3. Applications

3.1. Synthetic Case

[24] The first case study is a synthetic case for a virtual urban catchment of 10 ha. Table 2 presents model parameters that are assumed to be known exactly. Two events are studied with one for calibration and the other for verification. The rainfall event for calibration has a uniform profile for 10 h with intensity of 5 mm/h. The rainfall event for verification has a triangle profile with a maximum intensity of 10 mm/h, and it also lasts for 10 h. Both events are recorded with 1 min time step. The two rainfall events are fed into the conceptual rainfall runoff model described in section 2.1 with parameters in Table 2. The real system responses are identified as 1 min runoff time series. However, real values of rainfall and runoff cannot be exactly observed due to measurement uncertainty. The measured values are thus randomly generated assuming that each of them has a normal distribution with the mean being the real value and the relative standard deviation being arbitrarily set to 0.1. The two events are respectively shown in Figure 1 with the real and measured rainfall and runoff time series. The measured values are used in model calibration to estimate model parameters and to make predictions. A merit of using a synthetic case is that the known real data facilitate the testing and comparison of the three different methods.

Table 2. Assumed Model Parameters and the Search Range Used in Calibration in the Synthetic Case
Parameter SymbolsParametersSet ValueSearching Range
P1Initial loss (mm)0.5[0, 1]
P2Constant loss0.1[0, 0.2]
P3Reservoir constant K1 = K2 (min)10[1, 20]
P4Lag time (min)10[1, 20]
P5Base flow (L/s)10[0, 20]
Figure 1.

Real and measured rainfall and runoff in the synthetic case: (a) uniform event for calibration; (b) triangle event for verification.

3.2. Chassieu Catchment

[25] The Chassieu catchment is located in the east of Lyon, France. It is an industrial area that covers 185 ha. The imperviousness coefficient of this area is 0.72, and the runoff coefficient is around 0.4. In the hydrological model, a rough impervious area of 80 ha is assumed, and the imperviousness will be adjusted in calibration by the model parameter of constant proportional rainfall loss P2. The impervious area (roads, parking lots, roofs, storage and activity areas, etc.) is drained by a separate storm water sewer system. The pervious area is not connected to the sewer system. Two events in March 2008 are monitored to test the three approaches for uncertainty estimation in hydrological model calibration. Rainfall events are considered as independent if they are separated by at least 4 h of continuous dry weather (i.e., intensities less than 0.1 mm/h). The first event has an antecedent dry period of 9.6 h, and the second event has an antecedent dry period of 6.5 h. The rainfall time series is available with 1 min time step while the runoff time series is measured in 2 min time steps.

[26] The rainfall is measured by a 0.2 mm tipping-bucket rain gauge. The main uncertainty sources of the rainfall time series include errors in gauge measurements [e.g., Ciach, 2002], the conversion of tips into 1 min time series, and the representation of the areal rainfall by one rain gauge. In such a small-scale catchment, rainfall uncertainty is considered mainly from measurement errors rather than from spatial variability (the same assumption has been made by Balin et al. [2010]). This would be different for large catchments where the poor knowledge on spatial heterogeneity of rainfall may be the main source of uncertainty. An empirical evaluation of the relative standard deviation of uncertainty in rainfall measurements is 10%. This corresponds to setting inline image in equation (10).

[27] The runoff is not measured directly but computed from an analytical relationship with a combination of several measured values mainly including geometric parameters of the cross section of the channel, water depth, and mean flow velocity. Different sources of uncertainty are quantified either theoretically or empirically and are then propagated through the analytical relationship using the Law of Propagation of Uncertainties (LPU). Standard uncertainties in water depth and in mean cross section flow velocity in the catchment outlet sewer pipe are equal to 7.5 mm and 0.05 m/s, respectively. Resultant relative standard uncertainties in runoff measurements fluctuate with varied discharges, approximately ranging from 15 to 25% (refer to Bertrand-Krajewski and Bardin [2002] and Muste et al. [2012] for details).

[28] The hyetographs and hydrographs of the two events are displayed in Figure 2, respectively. The uncertainty intervals of runoff are also shown in the figure. Uncertainty in runoff discharges is significant, with typical orders of magnitude for measurements in sewer systems. Table 3 shows model parameter ranges that will be used as prior information in calibration according to the catchment information. The possible negative values of base flow consider the effect of water exchange with groundwater.

Figure 2.

Rainfall and runoff measurements in Chassieu: (a) event for calibration; (b) event for verification.

Table 3. Initial Parameter Ranges in Calibration in the Chassieu Case
Parameter SymbolsParametersSearching Range
P1Initial loss (mm)[0, 2]
P2Constant loss[0, 0.5]
P3Reservoir constant K1 = K2 (min)[1, 60]
P4Lag time (min)[0, 40]
P5Base flow (L/s)[−10, 30]

4. Results and Discussion

[29] The three approaches are tested for both the synthetic case and the Chassieu catchment. The dimension of the calibration problem in the second and third approaches depends on the choice of the number of rainfall multipliers. Different dimensions are investigated in both approaches.

4.1. Synthetic Case

4.1.1. Parameter Uncertainty Estimation

[30] The conceptual rainfall runoff model is calibrated with the uniform event in Figure 1a using the measured rainfall and runoff. The three approaches coded based on the original DREAM are performed respectively. The number of hydrological model simulations required to assure stable distributions of calibrated parameters was found to vary depending on the dimension of the calibration problem. In general, the larger the dimension of a problem, the more model simulations are required. In the second and third approaches, different numbers of rainfall multipliers between 1 and 200 are studied. The grouping strategies of rainfall observations are discussed in section 4.1.3. The number of model simulations in each calibration practice is identified by a trial and error approach. When the inclusion of more simulations does not significantly alter the posterior distributions of parameters, the chains are believed to be converged. The convergence of the chains also requires the Gelman-Rubin convergence diagnostics [Gelman and Rubin, 1992] to be smaller than 1.2, which is a typical value used to declare the convergence of MCMC chains [e.g., Schoups et al., 2010]. The first approach uses a maximum of 10,000 model evaluations, while the second and third approaches with 200 rainfall multipliers perform a maximum of 500,000 model simulations. Posterior distributions of calibrated parameters are obtained according to values evolved in the last 20% model simulations that constitute the chains. Uniform distributions are employed to generate initial populations for model parameters and rainfall multipliers. The initial uniform distribution reflects the absence of prior information about the mode of the posterior PDF of model parameters and rainfall multipliers.

[31] Due to the presence of uncertainty in inputs and outputs, model parameters are evaluated with accompanied uncertainty. Figure 3 displays uncertainty of model parameters represented by cumulative density functions (CDFs) using different approaches together with their real values. The second and third approaches employ 30 rainfall multipliers in Figure 3 (every 20 rainfall observations of 1 min time step share one multiplier, that is, one multiplier corresponds to a 20 min time interval). Different ways of accounting for input uncertainty in calibration lead to parameter uncertainties with different distributions. The first and third approaches provide relatively close evaluations in comparison to the second approach (see Figure 3). The biases in model parameter estimates of the second approach are related to the biases of rainfall error estimates, which will be shown in the latter discussion.

Figure 3.

Posterior CDFs of model parameters evaluated by the three approaches and the real values in the synthetic case.

[32] The posterior distributions of rainfall multipliers identified by the MCMC method in the second and third approaches are displayed in Figures 4a and 4b, respectively, by box plots that depict five statistical values, i.e., minimum, lower quartile, median, upper quartile, and maximum. The second approach generates multipliers that are generally higher than 1, whereas the third approach identifies multipliers that distribute around 1. Considering that the measured rainfall data are generated by adding normally distributed random errors on the basis of real values, the multipliers by the second approach are obviously overestimated when their effects are not directly included as random variables in the posterior distribution expression. It also explains why the model parameter posterior distributions from the second approach deviate from those in the other two approaches.

Figure 4.

Posterior distributions of 30 rainfall multipliers in the synthetic case by the (a) second approach, (b) third approach, and (c) third approach with model structural errors.

[33] An advantage of using a synthetic case is that the real random errors of rainfall observations are known, and they can be used to validate the result of uncertainty evaluation for rainfall inputs. In Figure 4, the real rainfall multipliers are also displayed in comparison to those obtained by calibration. One real rainfall multiplier (representing a group of 20 observations in these cases in Figure 4) is obtained by averaging the real errors of the corresponding 20 observations. Rainfall errors evaluated by the third approach well agree with the real ones where all real errors are bracketed in the evaluated uncertainty intervals. In contrast, evaluations by the second approach are generally higher than the real errors. As a consequence, initial and constant losses are assessed higher in the second approach to compensate for overestimated rainfall intensities. Rainfall error estimates by the third approach capture the rough trend of real ones, however, with rather low precision. The reason is probably that the model is not sensitive to mild rainfall errors due to the low pass filter effect of the reservoir model (see section 4.1.3 for details). In this case, the estimates are slight modifications based on the prior initial knowledge of 10% rainfall errors.

[34] However, attention should be paid to the model structural uncertainty that might have an influence on rainfall error estimation. Figure 4c shows an example of rainfall error estimates when model structural errors are present but not explicitly accounted for as the three approaches tested in this paper. In this case, model structural errors are synthetically generated by normal distributions with the standard deviation of inline image and first order autocorrelation coefficient of 0.9. They are then added to the measured runoff in Figure 1a. Figure 4 clearly shows that the estimated rainfall errors will compensate for structural errors if they are significant but not explicitly accounted for. As a result, rainfall error estimates are model dependent for such approaches. Nevertheless, structural errors are generally poorly understood and the structural error varies within different model structures. In real cases, an accepted method for handling structural errors remains to be established and thus signifies a challenge for future work. A deeper discussion of the influence of model structural errors on rainfall error estimations is beyond the scope of this paper.

[35] Figure 5 shows histograms of evaluated model parameters and their correlations by the third approach using 30 rainfall multipliers as an example. The real values of parameters are also shown. Model parameters obtained by other approaches do not show much visual difference and are thus not shown due to limited space. All model parameters end with relatively small ranges compared to the initial search spaces. The resultant small ranges bracket the real values of parameters in Table 2. The most frequent estimated initial loss (P1) does not correspond to its real value. This is likely due to its correlation with the parameter of lag time (P4), considering the fact that the model responses by an overestimated P1 can be compensated by a lower P4.

Figure 5.

Parameter uncertainty evaluated by the third approach with 30 rainfall multipliers in the synthetic case.

4.1.2. Model Performances in Calibration and Verification

[36] The 90% confidence intervals of simulated runoff in calibration by the third approach are shown in Figure 6. Results by the other two approaches look similar and are thus not shown. When identifying uncertainty in simulated runoff, the last 20% of runoff simulations in multiple chains by the MCMC method are the possible system responses, which consider both parameter and input uncertainties (if input uncertainty is considered in the method). The real system response is known for this synthetic case and is also displayed. On the basis of runoff estimations from last 20% of model simulations in multiple chains, the uncertainty in runoff observations is incorporated by simply adding it on the simulation results, which results in uncertainty intervals accounting for uncertainty in parameters, inputs, and outputs. These 90% confidence intervals are also shown in Figure 6. These intervals embrace most observations. The uncertainty intervals considering only parameter and input uncertainty have much narrower widths compared to those considering all uncertainty sources. It indicates that uncertainty in runoff observations dominates the other two uncertainty sources in this case. This is noticeable as, in this virtual case, relative standard uncertainties in rainfall and runoff are set both equal to 10%. All of the three approaches show good performance in calibration.

Figure 6.

Uncertainty intervals of the calibration event using the third approach in the synthetic case.

[37] The calibrated models by the three different approaches are verified using the triangle-shaped event. The posterior distributions of parameters obtained in calibration are propagated through the model. Regarding the input of rainfall, the first approach uses the measured rainfall, while the second and third approaches incorporate rainfall uncertainty by random generation of each observation according to its uncertainty distribution. The simulated and real runoffs by different approaches are displayed in Figure 7 as well as runoffs incorporating observation uncertainty. It seems that the uncertainty intervals by the first and third approaches behave well in the sense of embracing observations satisfactorily. The second approach significantly underestimates the outputs, which corresponds to the overestimated rainfall loss parameters compensating for overestimated rainfall multipliers in calibration. However, it is difficult to further compare between different approaches by visual observation.

Figure 7.

Uncertainty intervals of the verification event using the three approaches in the synthetic case.

[38] It is worth mentioning that results by the first and third approaches are rather close, including the model parameter uncertainty evaluations and runoff evaluations in calibration and verification. This is due to the minor contribution of input uncertainty to total uncertainty in this case. As a result, the traditional way of using the Bayesian method to evaluate uncertainty in model responses (the first approach) seems reasonable to some extent in some cases. A possible attractive advantage of the third approach over the first one is that it also provides uncertainty evaluation in rainfall measurements. The evaluated rainfall errors might be used to provide rainfall records with accompanied uncertainty estimates. It is important particularly in cases where rainfall measurements are not so reliable such as when using measurements by one rain gauge to represent the rain over a whole catchment. Recently concern has been raised in rainfall evaluation by using a backward rainfall runoff model [e.g., Kirchner, 2009; Heistermann and Kneis, 2011].

4.1.3. Contribution of Input Uncertainty to Total Runoff Uncertainty

[39] It is observed, in the proposed virtual case, that the uncertainty in output measurements dominates the total uncertainty from Figures 6 and 7, while uncertainty in model inputs (rainfall) appears to play a minor role. This can be explained by the fact that linear reservoir models are very efficient low pass filters: even high but not systematic and uncorrelated errors in rainfall with short time steps are smoothed in the rainfall-runoff process. The smoothing effect of the hydrological model is quantified in order to quantitatively compare different sources of uncertainty. Knowing the parameters of the conceptual hydrological model, the unit hydrograph that converts net rainfall to runoff can be easily identified. The runoff output at one time step can be written as the sum of products of net rainfalls and corresponding elemental values in the unit hydrograph:

display math(12)

where U(i) is the response flow at ith time step to a unit input of rainfall, n is the duration of unit hydrograph expressed as a number of time steps, inline image. Obviously the uncertainty in runoff due to uncertainty in rainfall is lowered by averaging rainfalls at several time steps (with weights U(i)). Uncertainty in runoff due to uncertainty in rainfall can be evaluated by the Monte Carlo method [Muste et al., 2012]. In this synthetic case, the length of the unit hydrograph is around 60 time steps (see Figure 8a).

Figure 8.

Contribution of input uncertainty to total output uncertainty in the synthetic case: (a) unit hydrograph; (b) filter effect of the hydrological model changes with autocorrelation in rainfall measurement errors under the assumption of uniform rainfall.

[40] To simplify the problem, uniform rainfall is assumed over time. If the relative uncertainty of net rainfall inline image is 0.1 (ignoring the initial loss), the relative uncertainty of runoff inline image with the unit hydrograph in Figure 8a will be 0.016 using the Monte Carlo method. The minor contribution of rainfall uncertainty in this case makes reasonable the traditional way of using model residuals to represent the lumped uncertainty (the first approach) in calibration.

[41] The above analysis is based on the assumption of errors in rainfall measurements being independent, which is the case in this study. The filter effects of the hydrological model are also investigated if the errors in rainfall measurements are autocorrelated. Assuming that the correlated errors can be expressed by a first-order autoregressive equation,

display math(13)

where inline image is the error in rainfall measurement xi, ρ is the first-order correlation coefficient, and νi have independent normal distributions with a mean of zero and a constant variance inline image. Letting the relative uncertainty of rainfall measurements still be 0.1, uncertainty in outputs due to rainfall uncertainty is quantified using the Monte Carlo method when ρ =[0.1:0.1:1] under the assumption of uniform rainfall. Figure 8b displays relative uncertainties of runoff outputs with changing autocorrelations of errors in rainfall measurements. As expected, it is observed that the filter effect of the hydrological model becomes less significant with more autocorrelation involved. However, there is a mild tendency when the autocorrelation coefficient is lower than 0.5. When the autocorrelation coefficient equals 1, errors in rainfall measurement become systematic and the relative uncertainty in runoff achieves the same value as that in rainfall.

4.1.4. Grouping Strategy for Rainfall Measurements

[42] In the second and third approaches, rainfall measurements (with 1 min time steps) are grouped to share multipliers in order to reduce the dimension of the calibration problem. When grouping rainfall measurements, a balance between the representativeness and the complexity of the problem should be found. Different numbers of multipliers are used in calibration and simulation performances of models with the parameter sets of maximum posterior probabilities are examined and compared. Model performances are evaluated by the root mean square errors (RMSE):

display math(14)

[43] Figures 9a and 9b show RMSEs of model simulations when models are calibrated with rainfall measurements grouped in different sizes by the second and third approaches. The maximum-likelihood performances by the first approach are also displayed for comparison. In calibration, model performances improve moderately with more rainfall multipliers by the second and third approaches. In comparison to the first approach, the two latter approaches have better calibration performances at the expense of involving more parameters (rainfall errors) in calibration. The second approach provides even more satisfactory calibration performances than the third. However, the exclusion of the direct influence of rainfall uncertainty (as random variables) in the posterior distribution expression by the second approach leads to bias in model parameter estimation. As a result, the verification (Figure 9b) provides poorer results for the second approach in comparison to the other two approaches, which is consistent with the observation in Figure 7. Figures 9c and 9d display estimates of the two most sensitive model parameters P1 and P2 as a function of grouping numbers by different approaches. It is clear that the verification performances are closely related to estimates of model parameters. The estimates of model parameters by the second approach are grouping dependent, due to grouping-dependent estimates of rainfall errors, which vary more freely when their effects are not directly included as random variables in the posterior distribution. The first and third approaches have similar verification performances because rainfall measurement errors can only be effectively estimated in calibration mode and they identify similar model parameters.

Figure 9.

Grouping strategies for rainfall measurements with maximum likelihood parameter sets: (a) RMSE of model simulations in calibration; (b) RMSE of model simulations in verification; (c) estimated parameter of initial loss; (d) estimated parameter of constant proportional loss.

[44] Regarding the grouping strategy for rainfall measurements, with the third approach, around 30 measurements in a group (calibration and verification place the same number of measurements in a group) appear to be an appropriate number in this case because fewer measurements placed in a group does not seem to improve model performances significantly. Considering that the runoff at each time step is a function smoothing around 60 rainfall measurements (see section 4.1.3), grouping measurements over a time interval longer than the duration of the unit hydrograph does not provide enough flexibility to adjust errors in rainfall. On the other hand, excessive flexibility does not consistently improve calibration results and might lead to overparameterization. Based on experience of these cases, we suggest that the number of measurements in a group could be chosen to correspond to around 0.2–0.8 times the duration of the unit hydrograph.

4.2. Rainfall Runoff Modeling in Chassieu, France

4.2.1. Parameter Uncertainty Estimation

[45] The conceptual rainfall runoff model is calibrated with the calibration event using different approaches. The number of maximum model simulations is between 50,000 and 500,000 depending on the dimension of the calibration problem. Figure 10 shows the posterior CDFs of model parameters by different approaches. The second and third approaches group every 30 measurements to share one rainfall multiplier (which corresponds to a total number of 26 multipliers). Similar to results of the synthetic case, the parameter distributions by the first and third approaches are close and those by the second approach significantly deviate. Figure 11 shows the box plots of posterior distributions of rainfall multipliers (26 multipliers) by the second and third approaches, respectively. An interesting observation is that the two approaches identify the highest and lowest rainfall multipliers corresponding to the same time steps, which means that the two approaches provide consistent indication of the rough profile of the measurement errors. The estimated input measurement error can be partly attributed to the model structure error that is not explicitly considered in this paper.

Figure 10.

Posterior CDFs of model parameters evaluated by the three approaches in the Chassieu case.

Figure 11.

Posterior distributions of 26 rainfall multipliers in the Chassieu case by the (a) second approach and (b) third approach.

[46] However, multipliers by the second approach distribute more widely. This means that rainfall uncertainty can be overestimated without considering its direct impact as random variables in the posterior distribution expression. The direct incorporation of input errors as variables in the posterior distribution expression constrains the search for possible input errors in smaller areas. The considerably different CDFs of model parameters by the second approach with respect to the other two approaches can also be explained by the overestimated uncertainty in rainfall. The similar CDFs of model parameters by the first and third approaches again suggest that rainfall uncertainty plays a minor role in total output uncertainty in this case.

4.2.2. Model Performances in Calibration and Verification

[47] Figures 12a and 12b show the 90% confidence intervals of estimated runoffs in calibration and verification, respectively, for the third approach. Most measured runoff values are in the 90% confidence interval in calibration by visual observation. In verification, the first few runoff observations (in 40 min) fall out of uncertainty intervals. The overpredicted performances at the beginning of the event might be due to the fact that a constant initial loss in the conceptual model is too simple to model the real situation. The initial losses of the calibration and verification events are too different to be modeled by a single parameter value. This issue of using a more complex rainfall loss model will be further investigated in future studies. In this real catchment, uncertainty in runoff observations dominates other uncertainty sources in both calibration and verification.

Figure 12.

Model simulation results in calibration and verification by the third approach in the Chassieu case: (a) calibration; (b) verification.

[48] The models with optimal parameter sets of maximum posterior probabilities for different approaches are evaluated with their RMSEs. Figure 13 displays the RMSEs of calibration and verification performances. Similar conclusions can be drawn as in the synthetic case: the third approach provides slightly better calibration performances in comparison to the traditional first approach. These two approaches derive comparable verification results because rainfall errors can only be estimated in the calibration mode. In verification, rainfall errors of normal distributions with relative standard deviation of 10% are propagated. Again, the second approach offers good calibration performances but estimates model parameters with bias due to overestimated rainfall measurement errors. As a result, the model performances in verification are inferior by the second approach. The verification performances by the first and third approaches are not significantly different. It confirms again that the traditional way of using model residuals to represent the lumped uncertainty in calibration is reasonable, at least for some cases with small catchments where spatial variability of rainfall inputs is not the main issue and with short time steps.

Figure 13.

RMSE of model simulations with maximum likelihood parameter sets by different approaches for the Chassieu case: (a) calibration; (b) verification.

[49] Figure 13 also displays model performances with different numbers of rainfall multipliers using different approaches. The general conclusions for the comparison between different approaches in the synthetic case hold for this Chassieu case. In the third approach, model performances in calibration improve with decreasing size of measurement groups. The duration of the unit hydrograph in this case is around 200 min, combining the grouping strategy proposed from the synthetic case that a group should contain measurements over time intervals corresponding to around 0.2–0.8 times the duration of the unit hydrograph, and 40–160 rainfall measurements should be placed in a group. When fewer (than 20) measurements are placed in one group, further declining group sizes do not significantly change calibration results. The grouping strategy derived from the synthetic case seems to be adequate for the Chassieu case.

5. Conclusions

[50] Calibration is an essential step before confidently using a conceptual rainfall runoff model in various applications. The assessments of parameter and predictive uncertainty of hydrological models are important and uncertainty analysis is usually based on model calibration. In traditional approaches, uncertainty in parameters is mainly attributed to output errors and the errors in input are not explicitly addressed. However, this is believed to be unrealistic for real-world applications because measured forcing data (rainfall measurements in hydrological process in this case) often contain significant uncertainty. Recent research has put efforts into incorporating rainfall errors in calibration based on the Bayesian method. This paper focuses on the comparison of different approaches that differently treat rainfall errors in calibration and their applications to small urban catchments with a small temporal scale (1 -min time step). The abilities of different approaches to provide parameter estimations and model output quantifications are studied.

[51] Three approaches based on the Bayesian method are compared through applications to a synthetic case and a real case. The first approach does not consider rainfall uncertainty explicitly. The second approach treats rainfall uncertainty with rainfall multipliers but does not include their effect in the posterior distribution expression as random variables. The third approach employs rainfall multipliers and formulates the posterior distribution incorporating errors in both inputs and outputs, as initially proposed in the framework of BATEA by Kavetski et al. [2006a, 2006b]. All of the three approaches identify uncertainty in model parameters with relatively small ranges in comparison to the initial search space. The first and third approaches give relatively close evaluations on model parameter estimations in comparison to the second one. The second approach estimates parameter uncertainty with bias because rainfall multipliers are more easily evaluated with bias when their effect is not directly included in the posterior distribution as random variables. Consequently, the verification performance of output estimation by the second approach is poor. The third approach provides slightly better model performances in calibration, while the first and third approaches have similar model performances in verification because rainfall error estimation is only activated in calibration mode.

[52] In both cases (the synthetic case and the Chassieu case) analyzed in this paper, it appears that uncertainty in runoff observation is more significant than uncertainty in rainfall. The minor role of rainfall uncertainty makes the traditional way of using model residuals to represent the lumped uncertainty (the first approach) in calibration reasonable. This suggests the opposite to some previous findings that rainfall uncertainty has great influence on the runoff estimation results [e.g., Kavetski et al., 2006b; Vrugt et al., 2009]. To our knowledge, the alternative conclusions made in this study are mainly due to different temporal and spatial scales of the cases. The minor contribution of rainfall uncertainty is coherent with the fact that linear reservoirs are very efficient low pass filters: even high but not systematic and uncorrelated errors in rainfall at short time steps are smoothed in the rainfall-runoff process. The filter effect of the linear reservoir model is also investigated. The quantitative study in the synthetic case shows that relative uncertainty in rainfall can be moderated by the smoothing effect of the model, to around one order of magnitude lower. In this paper, only random and uncorrelated errors in rainfall intensity have been initially considered. If errors in rainfall measurements are correlated, the filter effect of the linear reservoirs will be less significant.

[53] The rainfall uncertainty (input uncertainty) is considered by multipliers in the second and third approaches. Several rainfall observations are grouped, and each group has one parameter to represent uncertainty. The number of rainfall multipliers which relates to the dimension of the calibration problem is studied as a factor that influences uncertainty estimations. Theoretically, each rainfall observation error can be viewed as an independent variable. But the computation complexity of the calibration augments dramatically as the dimension of the problem increases. A balance should be found between the representativeness and the complexity of the problem. An empirical grouping strategy is suggested to advise the appropriate number of measurements in one group. Considering that the runoff at each time step is a function smoothing several rainfall measurements, suitable flexibility should be provided by the grouping strategy to adjust errors in rainfall leading to errors in runoff. According to the experience of this study, we found that it is appropriate to group rainfall measurements over time intervals corresponding to around 0.2–0.8 times the duration of the unit hydrograph.

[54] It is worth mentioning that one possible advantage of the third approach over the first approach is that the estimated rainfall errors can provide an estimation of uncertainty attributed to rainfall records. The agreement of the evaluated rainfall uncertainty and the real errors in the synthetic case demonstrates that the approach can positively capture errors in rainfall measurements. However, attention should be paid to the model structural uncertainty, which is not explicitly considered in this study. Indeed, the estimated rainfall will compensate for structural errors if they are bot negligible. As a result, the rainfall error estimation is model dependent. The explicit separation of uncertainty in inputs and outputs in calibration provides a theoretically sounder basis for the step forward of treating different uncertainty sources.


[55] The authors wish to acknowledge the OTHU (Observatoire de Terrain en Hydrologie Urbaine-Field Observatory on Urban Drainage–– for the event records in Chassieu. The authors wish to thank J.A. Vrugt for providing the DREAM code. The authors also wish to sincerely thank the anonymous reviewers and the associate editor for their careful readings and constructive comments that significantly helped improve the quality of this work.