Improving particle filters in rainfall-runoff models: Application of the resample-move step and the ensemble Gaussian particle filter


  • Douglas A. Plaza Guingla,

    Corresponding author
    1. Laboratory of Hydrology and Water Management, Ghent University, Ghent, Belgium
    2. Department of Electrical energy, Systems and Automation, Ghent University, Ghent, Belgium
    • Corresponding author: D. A. Plaza Guingla, Laboratory of Hydrology and Water Management, Ghent University, Coupure Links 653, B-9000 Ghent, Belgium. (

    Search for more papers by this author
  • Robin De Keyser,

    1. Department of Electrical energy, Systems and Automation, Ghent University, Ghent, Belgium
    Search for more papers by this author
  • Gabriëlle J. M. De Lannoy,

    1. Laboratory of Hydrology and Water Management, Ghent University, Ghent, Belgium
    2. NASA Goddard Space Flight Center, Greenbelt, Maryland, USA
    3. Universities Space Research Association, Columbia, Maryland, USA
    Search for more papers by this author
  • Laura Giustarini,

    1. Department of Environment and Agro-Biotechnologies, Public Research Center - Gabriel Lippmann, Luxembourg
    Search for more papers by this author
  • Patrick Matgen,

    1. Department of Environment and Agro-Biotechnologies, Public Research Center - Gabriel Lippmann, Luxembourg
    Search for more papers by this author
  • Valentijn R. N. Pauwels

    1. Department of Civil Engineering, Monash University, Clayton, Victoria, Australia
    Search for more papers by this author


[1] The objective of this paper is to analyze the improvement in the performance of the particle filter by including a resample-move step or by using a modified Gaussian particle filter. Specifically, the standard particle filter structure is altered by the inclusion of the Markov chain Monte Carlo move step. The second choice adopted in this study uses the moments of an ensemble Kalman filter analysis to define the importance density function within the Gaussian particle filter structure. Both variants of the standard particle filter are used in the assimilation of densely sampled discharge records into a conceptual rainfall-runoff model. The results indicate that the inclusion of the resample-move step in the standard particle filter and the use of an optimal importance density function in the Gaussian particle filter improve the effectiveness of particle filters. Moreover, an optimization of the forecast ensemble used in this study allowed for a better performance of the modified Gaussian particle filter compared to the particle filter with resample-move step.

1. Introduction

[2] Every year, human and economic losses are reported all around the world due to the presence of floods. Therefore, the scientific community actively is investing in improving the current flood forecasting systems. Conceptual rainfall-runoff models are an important component in operational flood forecasting systems. Generally, these models represent the study area by a number of water reservoirs through which different inflows and outflows (for example, infiltration, evapotranspiration, discharge) interact dynamically. Examples of such models are the Hydrologiska Byrans Vattenbalansavdelning (HBV) [Lindström et al., 1997] model and the Probability Distributed Model (PDM) [Moore, 2007] or variations derived from these models. From a technical point of view, the simplicity of conceptual models is an advantage that offers flexibility in the implementation. However, the identification of the model parameters that lead to realistic model predictions is a complex task. Moreover, the uncertainties in the forcings, model parameters, and simplifications in the model physics affect the overall performance of the conceptual model [Kavetski et al., 2006]. One way to reduce the predictive uncertainty of conceptual hydrologic models is the use of data assimilation to regularly update models using externally obtained data sets [Vrugt et al., 2006; Moradkhani and Sorooshian, 2008]. Nowadays, sequential data assimilation is also a key component in flood forecasting systems. The study carried out in this paper contributes to the ongoing research of improving sequential data assimilation methods.

[3] Kalman [1960] developed the discrete Kalman filter, which is a square-error estimator for linear systems. In his seminal paper, Kalman used the state-space representation in order to generalize the application to any kind of linear system. It was possible to extend the application of the filter to different systems and to develop nonlinear versions from the original Kalman filter, such as the extended Kalman filter [Hoeben and Troch, 2000], unscented Kalman filter [Wan and Van Der Merwe, 2000] and the ensemble Kalman filter (EnKF) [Evensen, 1994]. The EnKF is one of the most frequently used assimilation methods in hydrology [Reichle et al., 2002]. One limitation in the EnKF application is the underlying assumption of Gaussian forecast and observation errors. In order to tackle this limitation, nonparametric filters such as particle filters have been developed.

[4] In the particle filter methodology, the posterior of interest is described by the point mass approximation allowing for the representation of any kind of distribution. In other words, the assumption of Gaussian distributions, which is held in the application of the Kalman filter, is relaxed when using particle filters. This method has been used to assimilate discharge records into conceptual rainfall-runoff models [Moradkhani et al., 2005; Weerts and El Serafy, 2006] and to assimilate water stage records into hydraulic models [Matgen et al., 2010; Giustarini et al., 2011]. This method has also been used for the assimilation of soil moisture data [Plaza et al., 2012], for the estimation of model parameters [Montzka et al., 2011], and the estimation of root-zone soil moisture conditions [Nagarajan et al., 2010]. All these studies share a similar implementation of the particle filter, which is known as the generic particle filter or the standard particle filter (SPF). The SPF simplifies the computation of the importance weights allowing for a straightforward implementation. However, this simplification could affect the overall performance of the particle filter, mainly when the observation error is small. In Weerts and El Serafy [2006], the EnKF and the SPF are intercompared, leading to the conclusion that the EnKF is more robust with respect to forecast and observation errors. Other studies using the particle filter are discussed in Leisenring and Moradkhani [2011], DeChant and Moradkhani [2012], Leisenring and Moradkhani [2012], and Liu et al. [2012].

[5] Recently, the SPF has been applied in combination with the Bayesian model averaging approach in order to update the model weight at each assimilation time step [Parrish et al., 2012]. In the same context of model selection, particle Markov chain Monte Carlo (MCMC) methods [Andrieu et al., 2010] have been used [Rings et al., 2012; Vrugt et al., 2012] in more sophisticated implementations of the particle filter. Moradkhani et al. [2012] reported an increase of the effectiveness of the SPF by using MCMC moves in a joint state-parameter estimation study.

[6] The main goal of this study is to conduct an exploration of two possible options that can lead to an improvement in the operation of the particle filter when state estimation is performed in rainfall-runoff models. More specifically, a resample step based on MCMC methods is included in the SPF in order to improve the spread of particles. The second alternative consists of the enhancement of the importance sampling step in the Gaussian particle filter (GPF) [Kotecha and Djuric, 2003a] by considering a posterior estimate from an EnKF to generate the importance density function. The characteristics of the proposed techniques are studied in a synthetic experiment where artificial discharge records are assimilated into a conceptual rainfall-runoff model. The methodologies are assessed by the assimilation of in situ observed discharge data. A comparison is carried out between the proposed techniques, the EnKF, and the SPF.

2. Theory

[7] In state estimation theory, the evolution of the simulated system states is represented as follows:

display math(1)

where inline image is a possibly nonlinear function (model) of the state vector inline image, the forcings inline image, and process noise inline image, with t as the discrete time index. The notation inline image represents the estimation a posteriori (after correction) at time step inline image is the estimation a priori (before correction) and inline image the estimation a posteriori at time step t.

[8] The update is performed when inline image is corrected by using the information from the observations, which are described by:

display math(2)

where inline image can be a nonlinear function of the current true state and observation noise inline image.

[9] The main goal in Bayesian filtering is to find or approximate the probability density function of the current state given the observations, i.e., the posterior inline image, where inline image indicates the sequence of observations inline image. The posterior can be obtained recursively in two steps.

[10] The prediction step:

display math(3)

and the correction step:

display math(4)

[11] In the prediction step (equation (3)), the prior inline image is obtained based on the fact that the transition inline image and the posterior at time step t – 1 are known. The transition is the probabilistic model of the system and is described by the process model (equation (1)). In the correction step (equation (4)), considering that a new observation at time t becomes available, the prior is corrected according to Bayes's rule by using the information from the likelihood distribution inline image.

[12] The optimal Bayesian solution (equations (3) and (4)) is difficult to determine since the evaluation of the integrals might be intractable. In this paper, approximate solutions, e.g., EnKF and particle filter, are treated.

2.1. Ensemble Kalman Filter

[13] The EnKF and the particle filter aim to approximate the posterior distribution by a set of random samples, hereafter referred to as ensemble members or particles. In the EnKF, the distributions are considered to be Gaussian, and therefore characterized by the mean and covariance. By using Monte Carlo (MC) integration methods, the covariance is approximated by the sample covariance. The EnKF method is presented in two steps.

[14] First, the state propagation represented by equation (1) can be extended for a probabilistic model governing the ensemble state evolution. Specifically, we assume that at time t, we have an ensemble of N forecasted state estimates with random errors.

display math(5)

with inline image the forecast ensemble state vector, i the ensemble member index, and N the size of the ensemble. The estimate of inline image is given by the ensemble mean:

display math(6)

and the ensemble state error matrix is defined by:

display math(7)

[15] By means of the MC approach, the forecast error covariance can be approximated by the sample error covariance as follows:

display math(8)

[16] As reported in Burgers et al. [1998], the observations inline image should be perturbed in order to assure sufficient spread according to:

display math(9)

with inline image a white Gaussian noise characterized by a zero mean and a covariance inline image. The matrix inline image should represent the uncertainty of the observations.

[17] The second step is the correction step where the Kalman gain has to be computed and the analysis is performed. Here, the approximation of the error covariances is used in order to determine the ensemble Kalman gain.

display math(10)

[18] Finally, the updated state ensemble is given by:

display math(11)

[19] Since inline image and inline image correspond to nonlinear functions, the method of Houtekamer and Mitchell [2001] is used in this study. This method simplifies the computation of the Kalman gain by approximating the terms inline image and inline image directly from the ensemble members as follows:

display math(12)
display math(13)


display math(14)

2.2. Particle Filtering

[20] Particle filters are sequential MC (SMC) methods that approximate the posterior by a set of random samples. In more detail, if we sample N independent and identically distributed random variables, inline image for inline image, then SMC approximates the posterior by the empirical measure.

display math(15)

where the sample representation is described by a mixture of Dirac delta functions and inline image denotes the Dirac delta mass located at inline image.

[21] At this point, drawing particles is unfeasible since the posterior is unknown. Nevertheless, it is viable to draw particles from a known proposal distribution (also called importance distribution). This forms the basis of the importance sampling principle. Sequential importance sampling (SIS) is the recursive version of the importance sampling MC method and the particle filters are based on the SIS approach.

2.2.1. Sequential Importance Sampling

[22] In SIS, the posterior is approximated by a set of weighted particles as follows:

display math(16)

where inline image are the normalized importance weights associated with the particles, which are drawn from the proposal distribution. Considering that the system state evolves according to a Markov process, and applying recursion to the filtering problem, the recursive expression for the not normalized importance weights is given by:

display math(17)

[23] The selection of the proposal inline image is important in the design stage of the SIS filter. The filter performance mainly depends on how well the proposal approximates the posterior. In Doucet et al. [2000], an optimal choice for the proposal density function is proposed

display math(18)

[24]  inline image is optimal in the sense that it minimizes the variance of the importance weights conditionally upon inline image and inline image. However, the application of equation (18) is complex from the implementation point of view. A common choice of the proposal is the transition prior function [Gordon et al., 1993; Kitagawa, 1996]:

display math(19)

[25] The choice of the transition prior as the proposal simplifies equation (17) resulting in an expression where the importance weights depend on their past values and on the likelihood inline image. A common choice of the likelihood density function is the Gaussian distribution that describes the misfit between the observation predictions and the observations, scaled by the (usually a priori defined) observation error.

[26] The consequence of not using an optimal proposal is that the variance in the importance weights increases, which degenerates the performance of the SIS filter in most cases. The large MC variation in the weights leads to a depletion of the particle set, which can be mitigated by the suppression of the particles with small importance weights and the replication of those with large importance weights. The latter is obtained by applying resampling with replacement to the particle set. Note that in SIS, the state variables are not updated, i.e., only the weights are updated.

2.2.2. Resampling

[27] Resampling is basically the selection and replication of the particles with high importance weights. This additional step to the SIS filter involves mapping the Dirac random measure inline image into an equally weighted random measure inline image.

[28] Gordon et al. [1993] proposed a methodology that consists of drawing samples uniformly from the random measure inline image with probabilities inline image. This is the basis of the sampling importance resampling method that is equivalent to multinomial resampling (MulR).

[29] Beside MulR, more efficient selection techniques in terms of a reduction of the resampled particles variance have been developed such as the stratified resampling (StrR) [Carpenter et al., 1999], systematic resampling (SysR) [Kitagawa, 1996], and residual resampling (ResR) [Higuchi, 1997; Liu and Chen, 1998]. For a theoretical description of the resampling strategies and their characteristics, the reader is referred to Douc et al. [2005].

[30] SysR is the widely accepted technique since the implementation is straightforward, minimizes the variance, and generally outperforms other approaches. A generic or standard implementation of the particle filter is composed of an importance sampling step with the transition prior density function as the proposal density followed by a resampling step with the SysR or StrR approach [Arulampalam et al., 2002]. Such implementation is referred to as the SPF.

[31] The additional resampling step mitigates the particle degeneracy problem. However, other problems referred to as particle impoverishment arise when the set of resampled particles collapses in the worst case to a single particle either due to a nonproper performance of the selected importance function (equal to the prior density function in the SPF) or due to the presence of too small observation noise. Another reason can be the wrong representation of the distributions due to an insufficient sample size. A way to deal with the impoverishment of the particles is by adding variability to the resampled particle set. This can be accomplished using resample-move algorithms discussed in the following section.

2.3. Resample Move

[32] An approach to mitigate the impoverishment of the particles is by applying a resample-move step to the resampled set [Gilks and Berzuini, 2001; Doucet et al., 2001; Fearnhead, 2002]. Resample-move consists of the application of MCMC along with SMC algorithms. MCMC methods are traditionally used when random samples from complex or multidimensional probability distributions are needed. The methodology consists of the construction of Markov chains through the generation of collections of correlated samples that approximate a target distribution.

[33] In the context of particle filters, the MCMC step is applied as a way to introduce particle variability and thus reducing the depletion of the resampled particles. The main idea is to construct a Markov transition kernel inline image of invariant distribution inline image with the following property:

display math(20)

[34] For this Markov kernel, if the resampled particles inline image are distributed according to the posterior then the new particle set inline image is still distributed according to inline image, with the additional fact that the obtained particle set might have more diversity. Even in the case when the set inline image is not distributed according to the posterior, the application of the MCMC step assures that the new set can only have a distribution closer to the posterior.

[35] In order to construct a Markov kernel, the Gibbs sampler or the Metropolis Hasting (MH) algorithms can be used. It is well known that the MH approach has an extra degree of freedom since this method allows for the sampling of the candidates according to some proposal and accept the candidate with the acceptance probability α. For the particular case of the SPF where the prior is identical to the proposal, the idea is to sample candidates from the transition prior and accept according to the following α probability:

display math(21)

[36] According to Doucet and Johansen [2009], the condition of ergodicity regarding the resample-move kernel is no longer required in order to be able to implement efficient recursive particle MCMC algorithms.

[37] In this paper, the SPF with MCMC is applied to a rainfall-runoff model in order to analyze the performance and compare it to other approaches. The SPF with MCMC move step is presented in Table 1. This algorithm is the result of the implementation of independent MCMC steps on each resampled particle along with the SPF. A possible drawback of the methodology is that a limited MCMC proposal comes with limited MCMC candidates to explore areas in the state space that could possibly lead to more accurate estimates.

Table 1. Particle Filter With Resample-Move Step
At time t = 0
• Sample inline image inline image
• Set the weights inline image
At time inline image
• Sample inline image
• Compute the weights inline image and normalize inline image
• Resample inline image to obtain N equally weighted particles inline image
MCMC move step: sample inline image.
– Sample inline image
– Sample the proposal candidates inline image
– Compute the acceptance probability inline image
– Set the state according to inline image

2.4. Improving the Importance Density Function

[38] The choice of the proposal distribution is one of the critical design issues in particle filters. A proper performance of the PF is expected when the following key assumptions are valid: the point-mass approximation should represent the posterior distribution adequately and the proposal distribution should approximate the posterior distribution as accurately as possible [Arulampalam et al., 2002].

[39] In case the first assumption is not completely valid, the MCMC move step has been proposed as a methodology to increase the spread of particles improving the resolution of the particle set and the corresponding point-mass representation of the posterior.

[40] For the second assumption, some approaches have been reported in the literature, e.g., the auxiliary particle filter (APF) [Pitt and Shephard, 1999], regularized particle filter (RPF) [Musso et al., 2001], and the unscented particle filter (UPF) [Van Der Merwe et al., 2001] among others, which are derived from these techniques.

[41] In the APF, approximated samples from the optimal importance density are obtained by using an auxiliary variable, whilst in the RPF, samples are obtained from a continuous approximation of the posterior rather than from a discrete density improving the performance of the resampling step.

[42] The UPF belongs to a set of techniques that approximate the optimal importance density by incorporating the current observation with the optimal Gaussian approximation of the state. In this context, the analysis statistics from extended Kalman filter and the unscented Kalman filter are valid approximations to the optimal proposal. In the UPF, the optimal proposal is approximated as follows.

display math(22)

[43] The samples inline image are drawn from a Gaussian distribution with mean inline image and covariance inline image given by the unscented Kalman filter and computed for every ith particle

[44] In the same line of optimal proposals, the EnKF [Evensen, 1994] has shown high efficiency in terms of accuracy and computational time demand as a nonlinear filter outperforming the extended and unscented Kalman filters in most cases. Therefore, a proper combination of the EnKF and the particle filter assures a higher performance over the SPF and EnKF. In the geophysical sciences, examples of this combination correspond to: the adaptive Gaussian mixture filter [Hoteit et al., 2008; Andreas et al., 2011], the weighted EnKF [Papadakis et al., 2010], and the particle Kalman filter [Hoteit et al., 2012] among others.

[45] In this study, we modify the structure of the GPF [Kotecha and Djuric, 2003a] by the inclusion of the EnKF to provide the importance density function. The GPF is selected as the particle filter structure based on some interesting features that are discussed below. The combination of the GPF and the EnKF is referred to as the ensemble GPF (EnGPF).

2.5. Ensemble GPF

[46] Kotecha and Djuric [2003a] introduced the GPF. Basically, GPF approximates the mean and covariance of the state vector involved in the estimation by using importance sampling. The strengths of this approach are: non-Gaussian and nonadditive noise applications, and unlike the SIS filter, resampling is not required. Due to the interesting features, the GPF structure is adopted in this study and the selection of the EnKF to provide the proposal distribution is the major contribution to the original algorithm.

[47] In GPF, the prior inline image and posterior inline image density functions involved in the correction step (equation (4)) are considered as Gaussian distributions. The considerations make it possible to simplify the computation of the importance weights. Moreover, the importance weights in the GPF methodology are directly obtained from the importance sampling approach unlike the SIS method where the recursive expression of the weights is used (equation (17)).

[48] In the EnGPF, the unnormalized importance weights are given by:

display math(23)

where inline image are particles drawn from the importance density function inline image and the parameters inline image, inline image,which are used in the approximation of the prior, are obtained from the transition prior density function inline image as follows:

display math(24)
display math(25)

where inline image represents the particle set obtained from the propagation of the particles through the nonlinear model (equation (1)).

[49] Here, the EnKF is used in order to obtain the particles inline image along with the sample mean inline image and sample covariance inline image of the particle set. Therefore, the proposal distribution can be approximated as a Gaussian distribution as follows:

display math(26)

[50] After the computation of the importance weights (equation (23)), the posterior can be approximated as a Gaussian distribution:

display math(27)

where inline image and inline image correspond to the weighted mean and weighted covariance which are computed from the particle set as follows.

display math(28)
display math(29)

[51] From the implementation point of view, the approximation of the posterior involves the replacement of the particle set, which is obtained from the application of the EnKF, by a new particle set that is generated according to a Gaussian distribution with parameters inline image. The generation of the new particle set can be seen as a particle-move step with the particles moved to more interesting areas of the state space. The move step might introduce variability to the particles avoiding the problem of particle impoverishment, thus eliminating the need of a resampling stage. Moreover, since the importance weights do not depend on their past values, the filter does not suffer from particle degeneracy.

[52] The EnGPF algorithm is presented in Table 2. A limitation of the filter performance could arise when the propagation of the mean and covariance is insufficient for the approximation of the posterior. However, the representation of the posterior by finite Gaussian mixtures overcomes this limitation by the propagation of higher moments of the distribution [Kotecha and Djuric, 2003b].

Table 2. Ensemble GPF
At time t = 0
• Sample inline image inline image
• Set the weights inline image
At time inline image
• Sample inline image
• Compute the sample mean and sample covariance.
inline image
• Compute the Kalman gain inline image where:
inline image
inline image
inline image
• Update the particles inline image where
inline image
• Compute the mean and covariance from the updated particle set:
inline image
• Compute the weights inline image and normalize inline image.
inline image
• Compute the weighted mean and covariance:
inline image
• Approximate the posterior to a Gaussian inline image

[53] In this study, we assume that the introduction of an approximated optimal proposal can improve the overall performance of the GPF, thus outperforming the standard EnKF. The assumption is validated by a synthetic experiment and a study with in situ observed data.

3. Material and Methods

3.1. Site and Data Description

[54] The study site corresponds to the Zwalm catchment in Belgium. Figure 1 shows the location of the catchment. Some characteristics of the catchment are: the drainage area is 114 km2, the maximum elevation difference is 150 m, the average annual temperature is inline image, the average annual rainfall is 775 mm, and the annual evaporation is approximately 450 mm.

Figure 1.

The location of the catchment.

[55] Meteorological forcing data with a daily resolution (same as the model time step) from 2006 and 2007 were used. The climatological station located in Kruishoutem provided the precipitation needed by the model. Potential evapotranspiration was calculated with the Penman-method using the station observations of air temperature, humidity, radiation, and wind speed. Daily discharge values at the outlet of the catchment were available for the entire study period.

3.2. Model Description

[56] A modified version of the HBV model, which is developed and explained in Lindström et al. [1997] is used in Matgen et al. [2006]. In this study, a simplified version of the HBV model is adopted in order to be able to evaluate the performance of the filters.

[57] Figure 2 shows a schematic of the hydrologic model with the catchment represented by three reservoirs: a soil reservoir, a fast reacting reservoir, and a slow reacting reservoir. The slow flow unit characterizes the water that flows through the ground and eventually ends up in the discharge point. The fast flow unit represents the water that flows directly into the discharge point. In Figure 2, the arrows represent the different modeled flows and the rectangular boxes correspond to the water storages.

Figure 2.

A schematic overview of the rainfall-runoff model.

[58] The equations in discrete time governing the water mass balance in the reservoirs are presented as follows.

display math(30)

where inline image (s) is the model time step, t (s) is the discrete time index, and inline image in m3 are the states of the system. Rtot is the total precipitation in m3/s and Etr the actual evapotranspiration in m3/s, which is computed based on the potential evapotranspiration Etp (m3/s).

[59] The simulated flows such as the actual evapotranspiration Etr, the infiltration Rin, the effective precipitation Reff, the percolation Per, the fast reacting reservoir input Rfast, the output flow of the fast reacting reservoir Qfast, the slow reacting reservoir input Rslow, and the output flow of the slow reacting reservoir Qslow depend on the model states and model parameters. All these flows are given in m3/s.

[60] The linear/nonlinear relationships between the model variables are presented as follows:

display math(31)

[61] where inline image are dimensionless model parameters, Smax is the storage capacity of the soil reservoir (m3), P is the maximum percolation(m3/s), inline image is the storage capacity of the fast reacting reservoir (m3), and inline image (m3/s) and inline image (1/s) are model parameters.

[62] Normally, the unit hydrograph of the catchment is required in order to obtain the modeled discharge Qdis. In this study, the discharge is computed as the summation of the fast Qfast and slow Qslow flows since the model time step of 1 day is larger than the concentration time of the study site.

[63] Before the application of the filtering techniques, the model parameter values were identified by the shuffled complex evolution (SCE-UA) algorithm [Duan et al., 1993], with a calibration period that corresponds to 10 years of historical discharge data (1997–2006). Table 3 presents the identified parameter values.

Table 3. Model Parameters and Initial Conditions, Units, Identified Values, Noise Magnitude Used for Truth Generationa
ParameterUnitsValueError: Standard Deviation
  1. a

    Indicates a dimensionless parameter.

λ 577 inline image
Smaxm33,821,038 inline image
b 374 inline image
α 0.53 inline image
Pm3 s−143.92 inline image
inline image 10.84 inline image
inline image 0.34 inline image
inline imagem333,818,822 inline image
inline imagem3s−16.91 inline image
inline images−1 inline image inline image
ssoil (t = 0)m3 inline image0.5 × ssoil (t = 0)
sslow (t = 0)m3 inline image0.5 × sslow (t = 0)
sfast (t = 0)m3 inline image0.5 × sfast (t = 0)

[64] The hydrologic system described above can be represented as a state space model according to equations (1) and (2). The state vector is given by inline image with inline image the nonlinear model described in equation (30) and inline image representing the input forcings. The forecast noise inline image is defined by the presence of uncertainties in the initial conditions, driving forcings, and model parameters. The observations correspond to daily discharge measurements inline image with the observation model given by: inline image. We assume that the observations are affected by additive white Gaussian noise.

3.3. Experiment With Synthetic Data

[65] A synthetic discharge data assimilation study is performed. The experimental setup consists of the artificial generation of true discharge records through the application of additive and multiplicative Gaussian noise to the initial conditions, forcings, model parameters. A true discharge data ( inline image) record is calculated based on this artificial true state vector ( inline image).

[66] For the generation of truth, the initial conditions of the three water storages were estimated by using the in situ observed discharge data, as presented in Table 3. With respect to the error structure, initial state conditions were perturbed by additive white Gaussian noise with zero mean and the standard deviation corresponding to 50% of the nominal initial condition values (see Table 3).

[67] The errors that might have been introduced in the derivation of the input evapotraspiration are considered in this study through the perturbation of the evaporation time series by white Gaussian noise with zero mean and standard deviation equal to inline image.

[68] Precipitation is considered to be affected by multiplicative error by following the approach presented in Leisenring and Moradkhani [2011], where a lognormally distributed noise is utilized in the perturbation of the precipitation Rtot as follows:

display math(32)

with inline image the perturbed precipitation at time t, inline image is a variance scaling factor for precipitation data that is set to 0.30, and wR is white Gaussian noise with zero mean and standard deviation equal to 1.

[69] Additionally, the model parameters shown in Table 3 are perturbed with Gaussian noise with zero mean and standard deviation set to inline image times the nominal value for each parameter, respectively. inline image is the variance scaling factor for the model parameter set with a valid range between 0 and 5. Large uncertainty is considered for the errors of the identified model parameter values with inline image equal to 1. However, the inline image factor is scaled for each parameter based on a sensitivity analysis that was carried out in order to prevent unrealistic model simulations. The scaling factors are indicated in the last column of Table 3.

[70] The true states obtained from the process described above are used in the generation of the true discharge. Finally, the synthetic observations are obtained by the perturbation of the true discharge with lognormally distributed noise according to equation (32) with the variance scaling factor ( inline image) set to 0.25. Figure 3 shows both an ensemble forecast (see below) and the true states while Figure 4 shows the forecasted and true discharge.

Figure 3.

Ensembles of the forecasted and synthetic-generated true states: black solid line corresponds to the ensemble mean, dashed lines corresponds to the maximum and minimum ensemble members, dots correspond to the synthetic-generated true states, and the gray shaded area shows the 95% confidence interval. The same symbols are used in the remaining figures.

Figure 4.

Same as Figure 3, but for the true and forecasted discharge.

[71] The aim of the synthetic study is to assess the performance of the filtering techniques when retrieving the true states and true discharge. Synthetic observations inline image are assimilated by the filters at every daily model time step during the year 2007. The standard deviation considered in the measurement error is set to inline image (m3/s). The EnKF, the SPF, the SPF with the resample-move step (SPF-RM) and the modified GPF (EnGPF) are intercompared.

3.4. Ensemble Quality Control

[72] It is clear that the performance of any assimilation method depends upon a realistic generation of the state ensemble. In this sense, two approaches are used in this study aiming at a correct representation of the forcing, parameter, and model structure errors. The first approach concerns the identification of error magnitudes that remains constant along the simulation period [De Lannoy et al., 2006], while the second approach is based on a dynamic update of the error magnitudes [Leisenring and Moradkhani, 2012].

3.4.1. Constant Error Magnitudes

[73] The quality of the discharge ensemble is verified according to De Lannoy et al. [2006], where the ensemble spread (enspt), the ensemble mean square error (mset), and the ensemble skill (enskt) have to be computed first and at each time step t:

display math(33)

[74] In equation (33), inline image is the modeled discharge (m3/s) for particle i at time t and inline image is the corresponding observation of the discharge in m3/s at time step t. In order to have a large enough ensemble spread, on average the ensemble mean differs from the observation by a value that is equal to the time average of the ensemble spread. Therefore, the following expression should be true:

display math(34)

where inline image indicates an average over the simulation period. Furthermore, if the truth is statistically indistinguishable from a member of the ensemble, the following expression should be true:

display math(35)

3.4.2. Dynamic Update of the Error Magnitudes

[75] A procedure to update the error magnitudes during the assimilation cycles was introduced by Leisenring and Moradkhani [2012]. More specifically, ensemble spread is updated by varying the variance multipliers ξ (a.k.a. variance scale factors) at every assimilation time step. The ξ value is increased when the absolute bias is larger than the outer 95th percent uncertainty bound, and it is reduced when the bias is smaller than the outer 95th percent uncertainty bound. The procedure is indicated as follows:

display math(36)
display math(37)
display math(38)

where inline image is the absolute value of the mean model error, ubt is the partial uncertainty bound, inline image is the lower 95th percent uncertainty bound of the predicted observation, inline image is the upper 95th percent uncertainty bound of the predicted observation, and ert is the ratio of the model error to the partial uncertainty bound. Finally, variance scaling factors are corrected according to ert at each time step as follows:

display math(39)

3.5. Experiment With In Situ Observed Discharge Data

[76] In this experiment, a time series from year 2007 corresponding to in situ observed discharge data ( inline image) is used in the assimilation experiment. A predefined observation error with a standard deviation equal to inline image (m3/s) is considered in the study. The ensemble generation is performed according to a combination of the methodologies described in sections 3.4.1 and 3.4.2 and further discussed below.

4. Results and Discussion

[77] The skill of the assimilation methods in the estimation of the model states and output flow is assessed by the comparison of performance metrics related to the ensemble mean prediction and the ensemble spread prediction. The accuracy of the filters is verified by the root-mean-square error (RMSE) and the Nash-Sutcliffe efficiency (NSE) index while the percent bias (%BIAS) is used as a measure of precision. Moreover, the spread of the predicted ensemble is supervised by the normalized root-mean-square error ratio (NRR) index with NRR < 1 indicating too much spread and NRR > 1 indicating too little spread. A detailed description of these metrics is presented in Leisenring and Moradkhani [2011].

4.1. Experiment With Synthetic Data

4.1.1. Resampling Strategies and the Number of Particles

[78] The application of a resampling strategy is important in order to overcome problems related to the degeneracy of the particles. A sensitivity test of the performance of the particle filter with different resampling strategies is conducted. Specifically, the performance of the SPF along with the MulR, ResR, SysR, and StrR strategies is quantified through the computation of the discharge RMSE, averaged over 50 MC runs. The RMSE is computed between time series of the modeled discharge and the true discharge. The generation of the true discharge is explained in section 4.1.2. An observation noise variance of inline image (m3/s)2 and five particle sets (32, 64, 128, 256, and 528) are considered in the experiment.

[79] Table 4 presents the mean and standard deviation (over 50 MC runs) of the discharge RMSE and the computational time demand of each resampling strategy. According to Table 4, the performance of the particle filter improves when more particles are used in the approximation of the posterior. However, beyond 128 particles, the improvement with more particles becomes marginal. We selected 128 particles as a good trade off between accuracy and computational time demand.

Table 4. RMSE (m3/s) of the Simulated and True Discharge, Averaged (μ) Over 50 MC Runs, With Indication of 1 Standard Deviation (σ) and Averaged Computational Time Demand ( inline image) (s)
μσ inline imageμσ inline imageμσ inline imageμσ inline image

[80] Moreover, the RMSE values are close to each other when comparing the different resampling strategies, especially when the number of particles is above 128. The StrR approach performs slightly better in terms of the RMSE mean and computational time demand, thus we select the StrR as the strategy to be used within the SPF in this study.

4.1.2. Ensemble Generation: Constant Error Magnitudes

[81] A discharge ensemble with large enough spread is obtained by the identification of the noise parameters involved in the generation of the forecast error. For this, the magnitude of the noises used in the generation of the synthetic observations were increased. More specific, the standard deviation of the noise used in the perturbation of the initial state values was increased from 50% of the nominal values (for the generation of the truth, cf. Table 3) to 60% of the nominal values for the ensemble forecast. For the perturbation of evapotranspiration, the standard deviation of the white Gaussian noise is set to inline image and for the precipitation, inline image is set to 0.50. The variance scaling factor inline image of the model parameters is equal to 2.

[82] Although the magnitude of the noise errors was increased, the ensemble did not show sufficient spread. Therefore, the states were additionally perturbed by additive Gaussian noise with zero mean and standard deviation equal to inline image, where inline image is the variance scaling factor for the state vector error, and it is set to 0.10. The magnitude of the state errors is considerably lower than the magnitudes for the error in the initial conditions, parameters, and forcings. This partly assures that the Gaussian component in the structure of the forecast error is not dominant enough as to lead to biased performances in favor of the Gaussian filters. The corresponding ensemble verification measures for discharge are:

display math(40)

[83] Figures 3 and 4 show the ensemble mean, the 95% confidence interval (CI), and the maximum and minimum ensemble members for the states and the synthetic-generated true discharge, respectively.

4.1.3. Estimation of the True States

[84] Table 5 presents the performance metrics between the true states and the estimated states of the five data assimilation methods. In Tables 5-8, the open loop ensemble (without data assimilation) is used as a baseline with the purpose of comparison of the filter performances.

Table 5. Comparison of the Performance Metrics (RMSE (m3) %BIAS NSE and NRR) Between the Modeled and True States for the Optimal Ensemble Spread Scenario
B Filter inline image inline image inline image
Baseline inline image−21.670.231.35 inline image30.990.750.62 inline image−33.49−0.581.00
EnKF inline image−22.810.221.36 inline image11.830.850.72 inline image−10.810.731.11
SPF inline image− inline image1.620.850.66 inline image−11.240.721.10
SPF-RM inline image−22.750.231.36 inline image10.060.780.76 inline image−10.980.751.09
EnGPF inline image−22.730.221.37 inline image−14.920.840.95 inline image−8.860.741.20 Water Storage in the Soil Reservoir

[85] The left part of Table 5 corresponds to the performance metrics for the water storage in the soil reservoir Ssoil. With respect to the measures of accuracy, the variant of the SPF (SPF-RM) has the lowest RMSE value and the EnGPF has the highest value, with a short distance in magnitude between these two values. Moreover, the NSE index indicates that none of the filters improve the accuracy when comparing to the baseline run, and a slightly worse performance is observed for the ENKF and EnGPF. The same trend is seen in the column corresponding to the %BIAS, which is a measure of precision, all the filters perform slightly worse than the ensemble run without data assimilation. Finally, the ensemble spread index NRR shows very close values for all the assimilation methods and the ensemble run, indicating too little ensemble spread. The overall results indicate that the water storage in the soil reservoir is poorly estimated due to a weak influence of this state on the total output flow (low observability). Water Storage in the Fast and Slow Reacting Reservoir

[86] The performance metrics corresponding to the estimation of the water storage in the fast reacting reservoir Sfast are presented in the center of Table 5. In terms of accuracy, the EnGPF has the lowest RMSE and a slightly lower value of the NSE index than the EnKF and SPF which have equal NSE values. The variant of the SPF has the worst accuracy performance with the highest RMSE value and the lowest NSE value. The SPF performs the best for the %BIAS with the lowest value and the ENGPF performs the worst with the highest value. Nevertheless, EnGPF performs the best for the ensemble spread with the highest NRR value. At this point, it is difficult to draw an overall insight with respect to the filter performances, but in general, all the filters perform a remarkable correction of Sfast in terms of accuracy, precision, and ensemble spread. The inconsistencies in the performances metrics are due to the fact that observation model is highly nonlinear, and the noise used in the generation of the synthetic observations is different from additive Gaussian noise. However, the performance metrics allows for tracking the states and for checking possible unrealistic state trajectories.

[87] The performances metrics for Sslow are located in the right part of Table 5. The same trend of the performances metrics for Sfast is observed for Sslow with a strong correction according to the RMSE, %BIAS, and NSE metrics for all filters. This strong correction decreases the predictive state ensemble spread with the ENGPF, which has the highest value for the NRR index. On the other hand, the EnGPF has the least BIAS indicating a high precision.

4.1.4. Estimation of True Discharge

[88] Table 6 shows the performance metrics between the discharge observations and the modeled discharge and the computation time demand (CTD) for each filter. Three scenarios are considered in the assimilation of inline image regarding the generation of the initial ensemble. The first scenario corresponds to the optimal ensemble spread case (left part of Table 6), insufficient ensemble spread is considered in the second scenario (center of Table 6), and excessive ensemble spread is also considered. The statistic metrics of the ensemble, which are reported in the table, were obtained by the increase or reduction of the noise parameters inline image and inline image, which were multiplied by a factor of 0.5 for the scenario of insufficient spread and 1.5 for the excessive spread scenario. The aim of conducting the experiment with three scenarios is to verify consistency in the performance of the filters when the ensemble spread is altered.

Table 6. Comparison of the Performance Metrics (RMSE (m3/s) %BIAS NSE and NRR) Between the Modeled and True Discharge and Computational Time Demand (CTD (s))
FilterOptimal SpreadInsufficient SpreadExcessive SpreadCTD
inline image inline image inline image
inline image inline image inline image

[89] The EnKF performance shows consistency concerning the three scenarios with the best performance when the ensemble spread is increased and the worst performance for the insufficient spread scenario. The performance of the filter is indicated by the metrics in Table 6 with a reduction of the RMSE and %BIAS and an increase in NSE values. A wider ensemble spread improves the accuracy and precision of the EnKF filter.

[90] In terms of accuracy, the SPF performs better than the ENKF for the cases of insufficient and optimal ensemble spread. This is observed by comparing the RMSE and NSE values of the SPF to the values of the EnKF in Table 6. For the case of insufficient spread, the precision of the SPF is also improved. The better performance of the SPF over the EnKF is somehow expected since the setup of the experiment in this study is close to a non-Gaussian state estimation problem and the results are consistent with previous studies [Leisenring and Moradkhani, 2011; DeChant and Moradkhani, 2012].

[91] For the case of excessive spread, the SPF performance is deteriorated compared to the EnKF performance. This limitation in performance is related to the increase in the parameter and state error magnitudes and the fact that only the states in the SPF methodology are resampled. In this sense, Plaza et al. [2012] reported a malfunction of the SPF when the major source of uncertainty in the ensemble corresponds to parameter errors and the SPF is solely used for state estimation. A way to verify that the recombination of model states and parameter values is affecting the SPF performance is by checking in detail the performance of the estimated states. The inspection of the performance metrics for Ssoil and Sslow showed reasonable values, with the same trend as explained in section 4.1.3; thus, the performance metrics are not presented. However, unrealistic metrics were observed for Sfast. Table 7 presents the performance metrics for Sfast and for all filters. It is clear in Table 7 that the performance metrics related to the predictive ensemble mean for the SPF indicates a collapse in the estimation of Sfast. The overall performance of the SPF is affected by the wrong estimation of Sfast.

Table 7. Comparison of the Performance Metrics of Sfast for Excessive Ensemble Spread Scenario
Baseline inline image67.750.600.57
EnKF inline image28.050.820.57
SPF inline image14071.82−0.011.08
SPF-RM inline image4864.39−0.010.94
EnGPF inline image4.650.840.80

[92] With respect to the performance of SPF-RM, the results are consistent with the study in Moradkhani et al. [2012] for the optimal and excessive ensemble spread scenarios, where the performance of this particle filter is improved compared to the SPF due to an increase in the diversity of the particle set. The latter is observed when comparing the NRR values to the SPF with a decrease in the value of this index for the SPF-RM. The performance metrics corresponding to the estimation of Sfast in Table 7 shows an underestimation of the state. The scope of this study is limited to the state estimation problem. However, state-parameter estimation is recommended in order to increase the effectiveness of particle filters where resampling is performed. For the insufficient ensemble spread scenario, the SPF and SPF-RM show identical performance due to a good performance of the SPF which cannot be overcome by the SPF-RM.

[93] According to the performance metrics for Qdis shown in Table 6 and the performance metrics for Sfast presented in Table 7, the EnGPF has the best performance compared to the rest of the filters with the lowest RMSE and % BIAS values and the highest values for the NSE index. In this study, the errors in the model parameters and in the states themselves play an important role in the representation of discharge uncertainty. Taking this into account, the stable performances shown by the EnKF and EnGPF for the three scenarios can be attributed to the presence of large enough process noise as to perform an accurate correction of the discharge. On the other hand, the ability for state correction in the SPF and SPF-RM is diminished when the parameter and state errors are inflated. Based on the facts explained above, the EnGPF outperforms the standard implementation of particle filter and its variant with a resample-move step in this particular study case. In order to extend this finding, further research is needed with respect to the level of uncertainty associated to the model structure and the degree of improvement obtained when a state-parameter estimation is performed by the particle filters presented in this study. Ensemble Generation: Variable Variance Multipliers

[94] In order to determine a possible improvement in the performance of the filters (optimal ensemble spread scenario), the variance scaling factors inline image and inline image were dynamically updated at every time step according to equation (39). The upper bound of the ratio of the error ert (equation (38)) is set to 5, which is the maximum limit of inline image.

[95] Table 8 presents the performance metrics between the predicted discharge and the true discharge along with the computation time demanded by each assimilation method. Here, the performance metrics listed in Table 6 (optimal ensemble spread scenario) are compared with Table 8. The results indicate an improvement in the accuracy and precision of the EnKF and EnGP when comparing RMSE, %BIAS, and NSE values to those presented in the left part of Table 6. The performances of the SPF and its variant also shows an improvement when the variance multipliers are updated with a decrease in the RMSE values, a reduction of the %BIAS, and the increase in the NSE values. Additionally, the ensemble spread shows a remarkable improvement for all the filters with the NRR values all close or equal to 1.

Table 8. Discharge Estimation Performance Metrics When Using Variable Variance Multipliers

[96] In general, the application of the variable variance multipliers leads to better filter performances than the static variance multipliers. However, additional research is necessary to determine the optimal upper bound of ert. In this study case, the dynamic adjustment of the noise levels of the model states and parameters can reduce the efficiency of the performances of the SPF and SPF-RM. In fact, Table 8 lists a higher RMSE value compared to the RMSE value of the EnKF, while the opposite is observed in Table 6 (optimal ensemble spread scenario).

4.1.5. Computational Time Demand

[97] The last column of Tables 6 and 8 shows the computer time demanded by each algorithm. The application of the EnKF involves the computation of matrix operations, while in the SPF the computation of the particle weights along with the resampling of particles is required. Although the EnKF and the SPF are based on different theoretical foundations and the corresponding implementations, both filters perform a similar computational efficiency.

[98] Moreover, the SPF-RM demands more computer time and the EnGPF slightly more compared to SPF and EnKF. This can be explained by the complexity of these filters. For the SPF-RM, the additional computer time demanded by the implementation of the RM step, which involves the generation and selection of a new set of particles, decreases the efficiency of the filter.

[99] The implementation of the EnGPF consists in the application of the EnKF and the GPF. According to Kotecha and Djuric [2003a], the GPF demands less computer time when compared to the SPF, since the application of the resampling step is not required in the GPF. The benefit obtained from this fact is that the EnGPF is computationally more efficient than the SPF-RM with a marginal increase in time demand when compared to EnKF and SPF. The efficiency of the SPF-RM can be increased by a selective application of the RM step as reported in Moradkhani et al. [2012].

4.2. Experiment With In Situ Observed Discharge Data

[100] The procedure adopted for the generation of the discharge ensemble is a combination of the two approaches presented in section 3.4. First, the noise levels inline image and inline image were calibrated in order to obtain the optimal spread with values of 0.05 for inline image and 0.6 for inline image. These values are considerably lower than those used in the generation of the synthetic observations (see section 3.3). Second, the noise levels are sequentially adjusted according to the variable variance multipliers approach.

[101] Figures 5 and 6 show the discharge ensemble mean, the 95% CI, and the maximum and minimum ensemble members for the EnKF in Figure 5 and for the particle filters in Figure 6. It is difficult to determine the best performance solely by visual inspection since all the figures show similar performance. However, a small difference is observed by checking the peak around time step 170. The EnKF, SPF, and SPF-RM performances show insufficient ensemble spread as to cover this peak flow. On the contrary, the ensemble corresponding to the EnGPF performance shows sufficient spread as to cover the peak flow around time step 170. The figures also show that the filters perform better for the low flows than for the high flows. This is consistent with the considered variance of the observation errors that depend on the magnitude of the observation at each time step (see section 3.5).

Figure 5.

EnKF performance for observation error with standard deviation equal to inline image.

Figure 6.

Performances corresponding to the particle filters for observation error with standard deviation equal to inline image.

[102] Performance metrics comparing the four data assimilation algorithms are listed in Table 9. Although all the values in Table 9 are close in magnitude, the performance metrics indicates that the data assimilation method with the least skill is the EnKF. With respect to particle filter performances, the SPF outperforms the EnKF and the SPF-RM outperforms the SPF and the EnKF. The lower noise levels considered in the real experiment allows for a correct state estimation performance in the SPF and SPF-RM. EnGPF has the best performance with the lowest value for the RMSE index and %BIAS index along with the highest value of NSE. Moreover, the NRR value is the lowest indicating an improvement also in the ensemble spread.

Table 9. Discharge Estimation Performance Metrics When Using Variable Variance Multipliersa
  1. a

    Real case scenario.


[103] Overall, the experiment with in situ discharge data demonstrate that the EnGPF can be applied to state estimation problems with certain degree of non-Gaussian noise. Nevertheless, further research is needed in identifying to what extent the EnGF is able to perform better or similar to natural non-Gaussian filters, such as the SPF and the SPF-RM.

5. Conclusions and Recommendations

[104] In this paper, two alternatives to improve the performance of the particle filter have been considered. The first approach consists of implementing a resample move step in the SPF structure, while the second approach consist of the combination of two nonlinear/Gaussian filters which are the EnKF and the GPF. The performances of the EnKF and the particle filters are assessed through experiments with synthetic discharge observations and in situ discharge data.

[105] In the synthetic experiment, the errors assumed in the control setup allows for an evaluation of the data assimilation methods in a non-Gaussian scenario or close to this scenario. In non-Gaussian scenario, the SPF should outperform the EnKF. However, the results showed a marginal improvement. Therefore, the assessment of the filters was extended to different error magnitudes. The EnKF considerably outperformed the SPF as a consequence of inflating the magnitude of the errors. This deficiency in the SPF performance was analyzed based on the skill of the filter in the estimation of the water storages in the three reservoirs. Results indicate a collapse in the estimation of the water storage in the fast reacting reservoir, and the cause is attributed to the recombination of the model states and model parameters performed by the resampling step. This finding leads to the recommendation that state-parameter estimation needs to be considered in further studies. The results obtained from the experiment with real data and concerning the performances of the EnKF and the SPF indicates an outperformance of the SPF.

[106] In general, the SPF with resample-move step shows a consistent performance. SPF-RM outperforms the standard implementation of the particle filter by dispersing the particle set after the resampling step. The additional RM step increase the CTD since extra particles are obtained from a second run of the rainfall-runoff model.

[107] The variant of the GPF, ENGPF outperformed the EnKF and the SPF in general, but ENGPF performed slightly better than the particle filter with resample-move steps in the real experiment. The good results corresponding to the EnGPF performance are attributed to the use of a better importance density function compared to the SPF and its variant. Additionally, the importance sampling step in the EnGPF does not involve resampling but the sampling of Gaussian distributed particles. The latter could lead to a divergence of the filter performance when the real posterior density distribution is different from Gaussian. However, the results of this study show that the EnGPF is able to deal with non-Gaussian error structure. The model used in this study correspond to a parsimonious rainfall-runoff model, and the concentration time is smaller than the model time step allowing for a simplification in the computation of the output flow. Further research is needed to extend the potential use of the EnGPF methodology to complex hydrologic models. In this context, the absence of resampling step in the EnGPF methodology allows for a straightforward parallel implementation of the algorithm that can be useful in the application to spatially distributed hydrologic models.

[108] Finally, the dynamic adjustment of the noise levels based on the accuracy of the mean prediction relative to the ensemble spread demonstrated the increase in the effectiveness of data assimilations methods. In this study, the initial ensemble spread before assimilation was optimized by the identification of the noise levels in order to assure enough ensemble spread as to cover the observations during the entire simulation period.


[109] The work in this paper has been funded mainly by the Belgian Science Policy for the HYDRASENS project in the frame of the STEREO II programme and partly by Secretaría de Educación Superior en Ciencia y Tecnología (SENESCYT). The first author would like to express his gratitude to Escuela Superior Politécnica del Litoral (ESPOL Guayaquil-Ecuador) for the support during the initial phase of his postgraduate studies. Gabriëlle De Lannoy was a postdoctoral researcher funded by the Foundation of Scientific Research of the Flemish Community (FWO-Vlaanderen). The authors are very grateful to the Associate Editor Hamid Moradkhani, the reviewer Nataliya Bulygina, and three anonymous reviewers for the valuable contribution to the development of this paper.