Forecast-informed low-flow frequency analysis in a Bayesian framework for the northeastern United States


Corresponding author: S. Steinschneider, Department of Civil and Environmental Engineering, University of Massachusetts Amherst, 130 Natural Resources Rd., Amherst, MA 01002, USA. (


[1] Structured variation in the frequency spectrum of critical hydrologic variables can have important implications for the design and management of water resources infrastructure, yet traditional hydrologic frequency analysis often ignores the influence of exogenous factors that can both precede and exert control over hydrologic responses. Moreover, emerging literature that has addressed predictable low-frequency oscillations in the probabilistic nature of hydrologic variables has focused almost exclusively on flood flows. This study explores a new approach for conditioning the frequency spectrum of hydrologic extremes on seasonal predictors and applies the method to annual minimum 7 day low flows, a critical low-flow statistic often utilized in water quality management and planning. A semiparametric local likelihood method is used to condition quantile estimates of the 7 day low flow on year-to-year hydroclimatic forecasts for two major rivers in the northeast United States. The local likelihood approach is employed in a Bayesian framework in which regional information is used to inform prior distributions of model parameters. The method is compared against a baseline approach that applies a static Bayesian inference with noninformative priors to derive unconditional parameter and quantile estimates. The implications of the approach for the efficacy of water quality regulations and as an adaptation to climate change are discussed.

1. Introduction

[2] The current theory of water resources engineering assumes that the frequency of hydrologic variables critical to planning and management follow an underlying, time-invariant stochastic process that can be estimated and utilized for risk assessment in engineering problems. Advances in hydroclimatic science over the past two decades have made it increasingly clear that this theory oversimplifies the probabilistic characterization of hydrology. Low-frequency climate oscillations and interannual shifts in antecedent watershed conditions (e.g., soil moisture, snowpack, etc.) have been shown to directly modify the likelihood of river flows critical to water management [Georgakakos et al., 1998; Grantz et al., 2005; Sankarasubramanian et al., 2009; Gong et al., 2010; Pui et al., 2011]. The effects of climate and watershed precursors on streamflow are sometimes predictable, particularly at a seasonal time step, and are even extendable to the frequency, magnitude, and duration of extreme hydrologic responses [Hirschboeck, 1988; Mosley, 2000].

[3] While the vast majority of research has focused on the predictability of flood flow processes [Cayan et al., 1999; Olsen et al., 1999; Jain and Lall, 2000, 2001; Kwon et al., 2008], recent work has found similar possibilities for extreme low flows as well [Kiem and Franks, 2004; Verdon-Kidd and Kiem, 2010; Steinschneider and Brown, 2011]. The predictability of low-flow events introduces the potential to produce year-to-year, forecast-informed estimates of critical low-flow statistics that could have significant implications for water quality management practices. In this paper we present a new approach to predict low-flow variability with hydrologic forecasts and apply the method to critical low flows in two major river basins in the northeast United States. The approach couches a semiparametric local likelihood method in a Bayesian framework to develop forecast-informed quantile estimates of annual minimum 7 day low flows (SDLFs), along with their associated error.

[4] A description of the frequency of critical low-flow statistics is vital to an accurate risk assessment of water quality degradation. Stochastic models of critical low flows are used to determine the T-year low-flow event for rivers and estuaries that dilute contaminants from point and nonpoint sources, allowing for the effective design of regulations that preserve water quality under sensitive flow conditions. For instance, total maximum daily loads (TMDLs) are often designed using the 10th percentile of 7 day low flows (i.e., the 7Q10) so that water quality standards will on average be violated only for the week of lowest flow once in every 10 years. Water quality trading markets are often based around contaminant concentration limits at the outlets of major rivers that depend on estimates of the 7Q10 at those locations. Despite their critical importance to the development of water quality regulations, there is a relative paucity of research regarding the frequency distributions of low flows, especially in contrast to similar efforts dedicated to flood flow analysis [Vogel and Wilson, 1996]. Those studies that have explored low-flow frequency analysis have met with limited success [Smakhtin, 2001]. In part this is due to the difficulty of identifying scalable models that can capture the true probabilistic behavior of extreme low flows through both space [Vogel and Kroll, 1990] and time [Saunders and Lewis, 2003].

[5] As water quality continues to become a concern in rivers across the United States, there is a growing need to better characterize the year-to-year risks of critical low flows. Currently, typical water quality regulations (e.g., regulated by the 7Q10) employ a static approach, basing requirements on historical statistics and enforcing the same level of treatment in each year, despite year-to-year variability in streamflows. Where there is strong interannual or decadal variability, this may lead to overly conservative regulations in some years and inadequately lax regulations in others. The potential effects of climate change, which may introduce trends in flow conditions, may further erode the utility of such static approaches to regulation [Lins and Slack, 1999].

[6] Recently, a number of studies have begun adapting frequency analysis procedures to nonstationary hydrologic time series, with a particular focus on flood statistics. The most popular approach has been to condition the parameters of a flood frequency distribution on different covariates through regression [Coles, 2001; Cox et al., 2002; Katz et al., 2002]. For example, Coles [2001] conditioned the location parameter of a Generalized Extreme Value (GEV) distribution for a flood series on time to model evolving flood risk. Frequency modeling with time-variant parameters has also been used to test for statistically significant trends and changing variability in flood series [Delgado et al., 2010], as well as for regional flood frequency modeling [Cunderlik and Ouarda, 2006; Leclerc and Ouarda, 2007]. Other methods for frequency modeling of nonstationary floods include the incorporation of trends into distribution moments [Strupczewski et al., 2001a, 2001b; Strupczewski and Kaczmarek, 2001], flood magnification and recurrence reduction factors [Vogel et al., 2011], and quantile regression [Sankarasubramanian and Lall, 2003]. A thorough review of many of these studies and others can be found in Khaliq et al. [2006].

[7] To the authors' knowledge, there have only been a handful of studies that have applied a nonstationary frequency analysis to drought statistics. For example, Burke et al. [2010] fit the parameters of a peaks-over-threshold model to monthly drought indices projected for the next century using global and regional climate models. These parameters were allowed to vary through time via regression on global mean temperatures, enabling the estimation of nonstationary extreme monthly drought events at different return intervals. Garcia Galiano et al. [2011] and Giraldo Osorio and Garcia Galiano [2012] used Generalized Additive Models for Location, Scale, and Shape (GAMLSS) to model the nonstationary behavior of annual series of maximum lengths of dry spells. Besides these studies, no other research was found that attempted to extend nonstationary frequency analysis to dry-period extremes.

[8] Two recent approaches in the flood flow literature provide particularly interesting avenues for further research and application to low flows. These include a Bayesian frequency analysis with time-variant parameters that depend on parametric regressions [El Adlouni et al., 2007; Kwon et al., 2008; Ouarda and El Adlouni, 2011], as well as a semiparametric local likelihood procedure [Sankarasubramanian and Lall, 2003]. The use of Bayesian methods to conduct frequency analyses with time-variant parameters is relatively new. A primary advantage of the Bayesian approach is that it provides an efficient estimation framework that allows prior knowledge of model parameters to be integrated into the analysis. For instance, El Adlouni et al. [2007] and Ouarda and El Adlouni [2011] conditioned the parameters of a GEV distribution of annual maximum precipitation on the Southern Oscillation Index (SOI), using a Bayesian prior distribution in a generalized maximum likelihood analysis [Martins and Stedinger, 2000] to inform the estimation of the shape parameter and restrict its values to a reasonable range. Various linear and quadratic regression models were explored to condition the location, shape, and scale parameters on the SOI. Kwon et al. [2008] employed a similar technique in a hierarchical Bayesian framework in which the prior distribution of the location parameter of a Gumbel density function was specified through a linear regression on several different hydroclimate covariates. One potential downside of these studies is the reliance on parametric modeling to relate distribution parameters to covariates. The identification of appropriate analytic functions can become very complicated when the relationship between the covariates and the original hydrologic data exhibit significant nonlinearity or heteroskedasticity.

[9] Semiparametric and nonparametric methods have been a popular approach to circumvent such difficulties in strict parametric modeling. Sankarasubramanian and Lall [2003] used a semiparametric local likelihood method to condition the frequency distribution of annual maximum floods on seasonal forecasts based on El Nino–Southern Oscillation (ENSO) and the Pacific Decadal Oscillation (PDO). They found that the semiparametric local likelihood approach could address nonlinearity and heteroskedasticity in the relationship between flood flows and predictors more flexibly than parametric methods, but suggested that Bayesian approaches should be pursued to better characterize quantile and parameter uncertainty.

[10] This paper synthesizes the two approaches described above and explores an application to low-flow hydrology. A variation of the semiparametric local likelihood estimation method is developed in a Bayesian framework to condition the fit of frequency distributions to SDLF series on seasonal hydrologic forecasts for two major rivers in the northeast United States. Prior distributions for model parameters are developed from regional information and used in the Bayesian estimation procedure to ensure statistically and physically meaningful posterior distributions. The hydrologic forecast for both rivers is developed using measures of oceanic circulation in the North Atlantic Ocean. A static Bayesian inference with noninformative priors is also considered for comparison. Estimates of the 7Q10 are then developed for each year in the record using both estimators in a “leave-one-out” approach, and the results are compared to highlight the benefits of the conditional fitting procedure.

[11] The rest of the paper proceeds as follows. The semiparametric local likelihood estimator and Bayesian modeling framework are introduced in section 2. Section 3 provides a brief overview of the method's application to two rivers in the northeast United States. Results are presented in section 4, and the study concludes with a discussion in section 5.

2. Forecast-Informed Low-Flow Frequency Analysis

[12] A formal approach is desired to condition the frequency analysis of low-flow statistics on preceding hydroclimatic predictors in order to better ascertain the year-to-year changes in the probability density function of critical low flows. The approach considered here utilizes a semiparametric procedure to define a likelihood function that can incorporate forecast information and flexibly account for nonlinearities and heteroskedasticity in the predictor-streamflow relationship. That likelihood function is then applied in a Bayesian framework that can account for prior information and uncertainty in model parameters and output. The application considered here accounts for seasonal hydroclimatic predictors; the approach can be generalized to include other predictors as well, such as long-term changes in land use and climate.

[13] Let Qt be a time-varying low-flow variable of interest (e.g., the 7 day low flow), with time t = 1, 2, …, N and N equal to the total number of observations. We assume that Qt is a random variable that follows a well-defined probability distribution conditional on a set of M predictors, Xt = {x1,t, x2,t, …, xM,t}, that vary through time. The conditional probability density function for Qt at time t is given by f(Qt|Θt, Xt), where Θt is a vector of parameters that will be implicitly conditioned on the predictors Xt. The term Xt is included in the density function as a reminder that the parameter set Θt will be conditioned on forecast information at time t, but Xt is not explicitly used in the evaluation of the density function. This is explained in more detail below. The goal of this study is to infer the pth quantile, Qp,t*, of the variable Qt* at some time t*, given a parameter set Θt* and predictor information Xt*. This can be achieved by solving the following integral for Qp,t*:

display math

2.1. Local Likelihood Method

[14] In order to solve for Qp,t* in equation (1), f(Qt*|Θt*, Xt*) must be well-defined. Several methods exist to estimate the conditional density function; this study focuses on a semiparametric local likelihood approach. First developed by Davison and Ramesh [2000], this semiparametric method utilizes a K-nearest neighbor (K-NN) weighting scheme and kernel smoothing to condition the maximum likelihood fit of the parameter set Θt* on predictor information for a frequency distribution of known functional form to the data. A detailed explanation of the semiparametric methodology is provided in Davison and Ramesh [2000] and Sankarasubramanian and Lall [2003]. A variation of this method is used in this paper and is reviewed here.

[15] The premise of the semiparametric local likelihood method is to embed forecast information from time t* into the parameter set Θt* of the conditional probability density function f(Qt*|Θt*, Xt*). An altered likelihood function is used in the estimation of the parameter set Θt* using all observations Qt=1:N,tt* except that at time t* (written more concisely in vector notation as Qt*). The observations Qt* contribute to a local likelihood function L(Qt*| Θt*, Xt*, Xt*) that is conditioned with forecast-informed weights w(Xt*, Xt*) = {w(Xt, Xt*)| t = 1, 2, …, N, tt*}. These weights are a function of covariates Xt* associated with each observation in the likelihood, as well as covariates Xt* measured at time t*. The local likelihood function is given by

display math

[16] The importance of the observation Qt to the local likelihood function increases as the weight w(Xt, Xt*) approaches 1. The opposite is true as w(Xt, Xt*) approaches 0, since math formula will approach 1 independent of the parameter set Θt*. Note that the notation for the conditional probability density function f(Qt|Θt*, Xt*) has been simplified to f(Qt|Θt*) in the likelihood function to emphasize the fact that the likelihood function is informed by forecast information strictly through the weights w(Xt*, Xt*). The parameter set Θt* is implicitly conditioned on the covariates as they modulate the weighted density values that compose the likelihood.

[17] The weights w are determined by a K-NN scheme that prioritizes streamflow observations with predictor values that are similar to that of Xt*. That is, a hydrologic observation at time t is given more weight in the likelihood function if Xt is similar to Xt* and less weight as Xt and Xt* diverge. A product form of the Epanechnikov kernel function is used to smooth the weights across observations symmetrically about the position Xt*:

display math

[18] Here, hm is the kernel bandwidth associated with the mth predictor. As xm,t* and xm,t converge, um,t approaches 0 and math formula approaches 1, giving more importance to the observation Qt in the likelihood function.

[19] The bandwidths hm for each of the M predictors control how many observations will be given weight in the likelihood function. Streamflow observations with associated predictors outside of the interval [xm,t*hm, xm,t* + hm] will be disregarded completely (i.e., given a weight of zero), while those in the middle of the interval will receive a weight near one. The bandwidth for each predictor can be set manually to ensure that a certain number of observations contribute to the likelihood function. Alternatively, bandwidths can be chosen via cross validation. The cross-validation procedure used in this study is described in section 2.3.

[20] In the original presentation of the semiparametric approach given in Davison and Ramesh [2000], the parameters Θt* were further conditioned on forecast information via a linear model. The linear model allowed estimates of distribution parameters to vary within the neighborhood of Xt*, assuming that the relationships between parameters and covariates were linear in that neighborhood. This study does not allow distribution parameters to vary within the neighborhood of Xt*. Rather, all forecast information is embedded in the weights used to adjust the likelihood function. This is equivalent to reducing the linear model used in Davison and Ramesh [2000] to a zero-order polynomial. This simplification was preferred because it is difficult to validate that a linear model (or any other polynomial model) accurately captures the relationship between distribution parameters and covariates in all local windows.

[21] A primary benefit of the local likelihood method over standard parametric approaches is its flexibility in dealing with nonlinearity and heteroskedasticity in the data. Using parametric approaches, a functional form of the relationship between distribution parameters and a predictor would have to be chosen a priori and implemented across the entire range of predictor values. This can prove difficult because nonlinear and heteroskedastic components in the predictor-predictand relationship are sometimes difficult to identify. That is not to say that parametric approaches could not model these features. However, it is often difficult for the modeler to elucidate these characteristics from the available data and decide upon one analytic relationship that would be appropriate across the entire data set range. As mentioned in Sankarasubramanian and Lall [2003], this problem becomes especially difficult when more than one predictor is considered. The local likelihood approach avoids this issue by incorporating the local structure of the predictor-predictand relationship in neighborhoods around the data point under consideration, allowing the data itself to drive the local model fit.

2.2. Bayesian Modeling Framework

[22] The local likelihood method can be set in a Bayesian framework that can account for prior information and quantify the uncertainty in model parameters and quantile estimates. In the Bayesian framework, previous knowledge about the parameter set Θt* can be incorporated into the estimation process through a probability density function P(Θt*) known as the prior distribution. This prior information is combined with a likelihood function, taken here as the local likelihood function in equation (2), to develop a posterior distribution of model parameters:

display math

[23] The integral in the denominator is a constant of proportionality required to ensure that the posterior distribution is a well-defined probability density function. The posterior distribution for model parameters can then be used to develop relevant statistics for the quantile estimate Qp,t* = F−1(p|Θt*). For instance, the expected value E[Qp,t*] can be calculated by integrating the product of quantile estimates F−1(p|Θt*) and the posterior distribution of Θt* over the entire parameter space:

display math

[24] When the prior distribution is conjugate to the likelihood function and the unknown parameter set is one-dimensional, a closed-form solution for the posterior distribution can often be developed. However, if the prior and likelihood function are not conjugate or the parameter set is multidimensional, the form of the posterior is often too complex to be solved using analytical methods. This challenge has been largely ameliorated with computational advances that enable the generation of large samples from the posterior distribution that can be used to empirically summarize any of its features. Markov chain Monte Carlo (MCMC) sampling provides a straightforward way to generate these samples. MCMC sampling simulates a random process that has the posterior distribution as its equilibrium distribution. There are many types of MCMC sampling algorithms available for use. In this study, the posterior distribution of model parameters is explored using the slice sampler [Neal, 2003] in the software package JAGS [Plummer, 2011]. The slice sampler is an efficient MCMC sampling algorithm that draws random samples from a statistical distribution by uniformly sampling from the region under the plot of its density function using horizontal “slices” taken across the density function. Further readings on the slice sampler and other computational methods for Bayesian inference are available in Carlin and Louis [2009].

[25] A unique benefit of the Bayesian framework is that the advantages of the flexible local likelihood procedure can be coupled with prior information to stabilize the estimation of Θt* and make it more robust. As mentioned in Sankarasubramanian and Lall [2003], the local likelihood method can be sensitive to the selection of bandwidth and quantile estimates can become unstable as more predictor variables are included and the dimensionality of the parameter space increases. Furthermore, the local likelihood approach effectively removes some of the available data from the calculation of the likelihood function when forecast-informed weights approach zero, thus reducing the sample size used for parameter estimation. Prior information can stabilize the parameter estimates with information separate from the at-site data, ensuring that the posterior distributions of parameter values are reasonable. The inclusion of informative priors will also reduce the uncertainty in parameter estimates. A regional analysis offers one source of information to aid in the development of prior distributions [Ribatet et al., 2007; Micevski and Kuczera, 2009]. The development of prior distributions from regional information for this study is discussed in detail in section 3.3.

2.3. Selection of Bandwidth

[26] The choice of kernel bandwidth hm for each of the M predictors has important implications for the fitting procedure. Small bandwidths can lead to highly variable fits, but larger bandwidths can cause excessive smoothing of quantile estimates. A natural choice for the bandwidth of each predictor in the Bayesian modeling framework can be obtained by maximizing the cross-validated log likelihood [Ramesh and Davison, 2002]:

display math

[27] Here, the posterior distribution of the model parameters math formula at time t is obtained when the tth observation is excluded from the data set and the posterior distribution of the parameters is evaluated using bandwidths h. That is, for every year in the record, the observation for that year is dropped from the data set but the forecast information Xt is retained, and the posterior distribution of model parameters is estimated using equations (2)(4) and bandwidths h. The log likelihood of the left out observation is then integrated over the posterior distribution of model parameters. The integrated log likelihood values are then summed across years to develop the cross-validated log likelihood. The set of bandwidths math formula that maximizes math formula are then used in the Bayesian local likelihood fitting procedure to estimate the conditional distribution of an unknown, future streamflow QN+1. The maximization of math formula can be computationally intensive, as the posterior distribution of model parameters must be reestimated for each year in the record across a range of bandwidths for each predictor. In the application considered in this study (which uses only one predictor), the choice of bandwidth for one river with 73 years of annual data took approximately 12 h on a high-speed personal computer when 25 different bandwidths were considered. Though not explored here, one possible approach for reducing the computational expense would be to use the original local likelihood approach of Davison and Ramesh [2000] for bandwidth selection before continuing with the Bayesian approach proposed in this study. However, the choice of bandwidth may vary significantly from that found using equation (6) because prior information is not present in the original local likelihood methodology. More research is needed to explore more efficient approaches for choosing kernel bandwidths in the Bayesian local likelihood approach.

3. Application of the Bayesian Local Likelihood Approach to Two Northeast United States Rivers

3.1. Data

[28] The frequency model presented in section 2 was applied to two major rivers in the northeast United States. Seventy-three years of daily streamflow data from 1 January 1938 to 31 December 2010 were gathered from United States Geological Survey (USGS) gauges on the Susquehanna and Connecticut Rivers (USGS gauges 01576000 and 01184000, respectively). These rivers were chosen for two reasons: (1) They span the extent of the Northeast and therefore ensure the method is representative across space, and (2) a water quality trading market is in place in both of their basins. Annual minimum 7 day low flows at each location were calculated for the summer (June–September) of each calendar year. The calculations of SDLFs were limited to the summer months to ensure that predictor variables precede SDLFs, which can on occasion occur in the winter, especially in northern rivers. Time series of the SDLFs for both rivers are presented in Figures 1a and 1b, along with a Lowess fit to the data.

Figure 1.

Time series of SDLFs, with Lowess fit, for the (a) Susquehanna and (b) Connecticut Rivers. Also shown are scatter plots of SDLFs and the springtime NAT index for the (c) Susquehanna and (d) Connecticut Rivers.

[29] The SDLF series for the Connecticut River exhibits significant nonstationarity, with a Mann–Kendall test suggesting a significant positive trend with a two-sided p value of 0.005. This trend is somewhat countered by a few, large SDLFs early in record, but a decadal pattern in the data is nevertheless clearly present. This makes the Connecticut SDLFs a prime candidate for testing how well the Bayesian local likelihood approach can capture nonstationary behavior. The stationarity condition of the SDLFs in the Susquehanna River is less clear. While the Mann–Kendall test finds no significant trends in the data, the Breusch-Pagan test [Breusch and Pagan, 1979] does detect a statistically significant change in the variance of the Susquehanna SDLFs through time at the 0.05 significance level. The ambiguity in the stationarity condition of SDLFs for the Susquehanna River is common among many hydrologic series across the United States [Stedinger and Griffis, 2011]. These data are useful in testing how the Bayesian local likelihood frequency analysis reflects changing risk, if any, in a hydrologic series whose stationarity is difficult to verify.

[30] Sixty-one years of daily streamflow data were also gathered for 189 USGS gauges along the eastern coast of the United States from 1 January 1950 to 31 December 2010. Time series of summertime SDLFs were calculated for all of these gauges. Only 28 of these rivers are used in the regional analysis to develop prior distributions (see section 3.3). These 28 streamflow gauges are considered “reference gauges” (i.e., relatively unaltered) according to the GAGES database [Falcone et al., 2010], and they are located north of 41°N latitude and east of 80°W longitude to ensure they are situated near the Susquehanna and Connecticut Rivers. Data from all 189 gauges were used only in the development of Figure 2 (described below). We note that the distinction between regulated and unregulated gauges is only made for the 28 gauges included in the regional analysis; no such distinction is made for the entire set of 189 gauges used to develop Figure 2.

Figure 2.

Pearson r correlation coefficients between summertime SDLFs at USGS gauging stations and both (a) the springtime NAT and (b) May flow averages. Correlations greater (less) than +0.25 (−0.25) are significant at the 5% significance level. The USGS gauging stations for the Susquehanna (red) and Connecticut (blue) Rivers are also shown.

[31] Previous work has found that a springtime (March–April–May (MAM)) pattern of sea surface temperature anomalies (SSTA) in the North Atlantic Ocean, called the North Atlantic Tripole (NAT) [Rodwell et al., 1999], exerts control over summertime streamflow processes in the Connecticut River Basin [Steinschneider and Brown, 2011]. A positive, springtime phase of the NAT has a tendency to persist into the following summer and cause storm tracks to shift further over land and deliver more precipitation to the river basin, while a negative NAT event has the opposite effect. The influence of shifting moisture delivery pathways forced by the NAT is not limited to the Connecticut River Basin, but is present in river basins throughout the Northeast (Figure 2a). This result suggests the NAT could be a useful predictor of summertime precipitation, and therefore SDLFs, in both rivers considered in this study. The relationships between the springtime NAT and SDLFs for both rivers are shown in Figures 1c and 1d. The relationship is significant for both rivers at the 0.01 significance level, with Pearson r values of 0.31 and 0.55 for the Susquehanna and Connecticut Rivers, respectively. The springtime index for the NAT used here is developed from average monthly North Atlantic SSTAs as described in Steinschneider and Brown [2011].

[32] In addition to the NAT, many studies have found that antecedent soil moisture can influence base flow volumes for several months, especially in the summer [Koster et al., 2010]. Average May flow volumes can be used as a readily available surrogate measure for basin-wide soil moisture conditions. While the springtime NAT is useful for predicting the likely summer rainfall that will enter regional river basins, May monthly flows are indicative of the moisture already stored in the basins entering the summer months. For the interested reader, we show the relationship between May flows and summertime SDLFs across the eastern coast of the United States (Figure 2b). These strong relationships indicate that low-flow predictability is not limited to the northeast United States, but is extendable to the mid-Atlantic and Southeast regions as well. This study only considers the NAT as a predictor of low flows in the Northeast, but future work could extend the analysis to southern gauges using May flows as a predictor.

3.2. Choice of Frequency Distribution

[33] In order to apply the conditional frequency analysis to river flows, a functional form of the conditional frequency distribution f(Qt*|Θt*, Xt*) must be chosen a priori. L-moment diagrams were used to facilitate this choice. L-moment ratio estimators are virtually unbiased for all distributions, making L-moment diagrams a robust method for choosing among alternative distributional hypotheses [Vogel and Fennessey, 1993]. These moment diagrams were used to select a distributional form for the unconditional frequency distribution f(Q) for both rivers considered. The functional form of the conditional frequency distribution f(Qt*|Θt*, Xt*) was then set to be the same as f(Q).

[34] Figure 3 shows L-moment ratio estimators for SDLF series for the two rivers considered, as well as the 28 gauges used in the regional analysis. Theoretical L-moment ratios are also included for several distributions often used for low-flow frequency analysis. The empirical L-kurtosis and L-skew of the Susquehanna and Connecticut rivers suggest that a three-parameter Weibull distribution can effectively model the frequency of both SDLF series. While the Weibull distribution may not be the best fit for the other 28 gauges, it is adequate. Therefore, a three-parameter Weibull model is considered sufficient for the purposes of this study, including the regional analysis. The probability density function of the three-parameter Weibull distribution is given by

display math

where κ > 0, γ > 0, and μmath formula are the shape, scale, and location parameters. The shape parameter controls the skew of the distribution, the scale parameter influences its statistical dispersion, and the location parameter acts as a lower bound for the data.

Figure 3.

L-moment ratio diagram for the SDLF series.

3.3. Development of Prior Distributions

[35] The development of prior distributions for κ, γ, and μ in the Bayesian local likelihood fitting procedure was based on a regional analysis of SDLF series at 28 gauges across the Northeast. Following a similar approach taken in Lee and Kim [2008], the three-parameter Weibull distributions were fit to each of these time series using standard maximum likelihood estimation (MLE). The shuffled complex evolution algorithm [Duan et al., 1992] was employed to perform the maximization because closed-form solutions are not available using the MLE method for all of the parameters of the Weibull distribution [Smith and Naylor, 1987]. Distributions were then fit to the 28 shape, scale, and location parameters and used as the priors in the Bayesian analysis.

[36] A lognormal (LN) distribution was found to adequately model the 28 fitted shape parameters, as shown in the probability plot given in Figure 4a. The prior distribution for the shape parameter of both the Susquehanna and Connecticut Rivers is given by

display math

where λκ = 0.322 and σκ = 0.214 are the MLE solutions for the mean and standard deviation of the fitted lognormal distribution.

Figure 4.

(a) Probability plot of the 28 shape parameters fitted in the regional analysis versus the theoretical quantiles of a lognormal distribution. Linear log-log relationships between drainage area and parameter estimates from the regional analysis are also shown for the estimated (b) scale and (c) location parameters.

[37] As shown in Lima and Lall [2010], the scale and location parameters of frequency distributions often exhibit a log-log linear relationship with catchment area. Figures 4b and 4c show the relationships between the logarithms of the 28 scale and location parameters and the logarithms of their associated drainage areas. Given this relationship, the prior distribution for the scale parameter for both rivers can be specified as

display math

where βγ,0 = −1.74 and βγ,1 = 0.96 are OLS estimates of the regression coefficients for the log-log linear relationship between the fitted scale parameters and drainage area, and DA is the log-transformed drainage area of the river basin of interest. The parameter σγ = 0.41 is set equal to the standard deviation of the regression residuals.

[38] Initially, a similar prior was developed for the location parameter. However, a lognormal prior distribution developed using a regression against drainage area leads to prior values of μ that are larger than some of the smallest SDLFs in the record for both rivers. This contradicts the nature of μ as a lower bound to the data. Setting μ to zero was also considered, but this significantly degrades the goodness-of-fit of the distribution to the data because the two rivers, and their associated SDLFs, are so large. To circumvent these issues and develop a prior that accurately reflects the location parameter as a true lower bound to the data, we set the prior distribution for μ for each river to a uniform distribution with lower bound 0 and upper bound equal to the smallest recorded streamflow value for that river, Qmin:

display math

[39] This approach is not quite satisfying because the prior is no longer completely independent of the data, but it is not clear how to develop a prior that can reconcile the relationship seen in Figure 4c with the requirement that μ be a true lower bound to the data. Nonetheless, the prior in equation (10) is sufficient to illustrate the methods proposed in this paper. The development of a more appropriate regional prior for a lower bound to hydrologic data is left for future research. We also note here that the development of all prior distributions in this study is rather simple and is primarily meant to illustrate the methods proposed in this study. More sophisticated regionalization procedures exist that could be extended to the development of regional priors and improve their specification [Stedinger and Tasker, 1985; Reis et al., 2005; Kjeldsen and Jones, 2009; Micevski and Kuczera, 2009].

3.4. Framework for Application and Comparison

[40] The Bayesian local likelihood (BLL) methodology was applied in a jackknife “leave-one-out” approach, in which each year's SDLF value was assumed unknown and the critical low-flow risk for that year was quantified with a 7Q10 estimate developed from all other available data. Here, the year left out corresponds to time t*, the associated SDLF and predictor values are given by Qt* and Xt*, respectively, and the 7Q10 estimate for the unknown year is denoted as Qp,t*. Bayesian inference was carried out using the slice sampler with 10,000 iterations to define the posterior distribution and a burn-in period of 5000 iterations. Three chains initially overdispersed were used in the sampling, and the Gelman and Rubin convergence factor was used to test for convergence [Gelman and Rubin, 1992]. Prior to the implementation of the methodology, the cross-validation procedure described in section 2.3 was used to determine appropriate bandwidths for each of the two rivers.

[41] To provide a baseline against which the methodology can be compared, a static 7Q10 estimate was developed with an unconditional, three-parameter Weibull distribution f(Qt*|Θbaseline,t*) whose parameters Θbaseline,t* are estimated using Bayesian inference with noninformative priors. The priors for the shape, scale, and location parameters were all set to noninformative uniform distributions with lower bound 0 and upper bounds 7, 10,000, and Qmin, respectively. The upper bounds for the shape and scale were made large enough to allow any reasonable parameter value to be adopted, and the upper bound for the location parameter was not changed from that in equation (10). With noninformative priors, the mean of the posterior distribution for model parameters approximates MLE parameter estimates, and the two will converge with large enough sample sizes [Freedman, 1999]. The likelihood function used in the baseline Bayesian inference is given by

display math

[42] Note that without forecast-informed weights, this likelihood function utilizes all of the SDLF data available. Employed in the jackknifing framework, the baseline Bayesian inference was used to fit an unconditional frequency distribution to all SDLF data besides that for year t*.The posterior distribution was used to produce an unconditional estimate of Qp,t* for each year. These estimates are expected to remain relatively constant from year to year, with variations arising only due to the effects of removing a given year from the fitting procedure. By developing the baseline in a Bayesian framework, comparable measures of uncertainty for parameter and quantile estimates can be developed and compared against the uncertainty in parameter and quantiles estimates from the BLL method. The baseline Bayesian inference was conducted with the same slice sampler, number of iterations, and burn-in period as the BLL approach.

[43] One final comparative run, termed the naïve-prior Bayesian local likelihood (naïve-prior BLL) approach, was also considered. In this run, the BLL approach was used as described in section 2, but the prior distributions developed from the regional analysis were replaced with the noninformative priors used in the baseline run. When compared to the BLL method, the naïve-prior BLL approach will help identify the effects that prior information has on the estimation procedure. Particular focus will be given to the influence of informative priors on estimation uncertainty.

[44] Three metrics were used to assess the performance of the BLL method. First, the year-to-year difference in 7Q10 estimates between the BLL, naïve-prior BLL, and baseline approaches were examined. The year-to-year differences highlight how the BLL method can express interannual changes in the risk of low flows. Second, the width of the 95% credible interval for year-to-year 7Q10 estimates was compared between the three approaches. This metric helps identify the precision of quantile estimates using the BLL approach, and it also highlights the isolated effect of prior information on estimation uncertainty. Finally, the percentage of times SDLF observations fall below 7Q10 estimates for each method was also examined. This statistic indicates whether the BLL approach is capable of accurately expressing the risk of critical low flows. In theory, this percentage should be around 10% for this application because the flow statistic of interest is the 7Q10.

4. Results

[45] In the BLL procedure, SDLF observations in all years besides the year of estimation, t*, were assigned weights in the likelihood function. These weights were calculated as a function of the distance between predictor values associated with those observations and the predictor value for the year t* (equation (3)). Figure 5a shows the neighborhood of SDLF values in the Susquehanna River that contributes to the local likelihood function when t* equals 1958. The Epanechnikov kernel is also included. Similar results for the Connecticut River are shown in Figure 5b. The cross-validated bandwidths used to determine the width of the neighborhoods equal 0.68 and 0.35 for the Susquehanna and Connecticut Rivers, respectively.

Figure 5.

Local likelihood fitting for the (a) Susquehanna and (b) Connecticut Rivers for the year 1958. The vertical solid line indicates the springtime NAT value during 1958 and the open points indicate SDLFs. The shaded area denotes the local neighborhood identified by the BLL method, and the light gray dotted line shows the local likelihood weights produced by the kernel.

[46] The local neighborhood determines which observations the MCMC sampler “sees” when trying to define the posterior distribution. In years when the predictor varies significantly from its value in 1958 and falls outside the neighborhood, the SDLF observation from that year is given a weight of zero, effectively setting the pdf value of that observation in the likelihood function to unity. This precluded Weibull parameters from being adjusted to make these observations more likely, because no matter the parameter values chosen, these observations have no influence on the estimation of the posterior distribution. Conversely, for years in which predictor values are very similar to the predictor in 1958 and fall in the middle of the neighborhood, observations from those years are given large weights, preserving their influence on the posterior distribution. The kernel defines how rapidly weights near unity at the center of the neighborhood decay toward zero at the neighborhood's edges. In Figure 5a, the neighborhood defined for the Susquehanna River in 1958 is quite broad, and the kernel decays very slowly from its center. This contrasts the narrower neighborhood for the Connecticut River that exhibits a kernel that decays more rapidly (Figure 5b). The size of the neighborhood and rate of decay of the kernel are a function of the bandwidth, but more broadly they reflect the strength of the relationship between the spring NAT predictor and SDLFs in the two rivers. When the relationship between the predictor and predictand is weak (Pearson r = 0.31 for the Susquehanna), the cross-validation produces a larger bandwidth because more data are needed to develop an accurate model fit to the data. For somewhat stronger relationships (Pearson r = 0.55 for the Connecticut), a better fit is possible with smaller bandwidths that allow the forecast information to more actively control which observations influence the likelihood function.

[47] Estimates of the posterior distribution for the 7Q10 were developed for each year of the 73 year record using the jackknifing approach. Figures 6a and 6b show the median values of these posteriors produced by the BLL approach. Median 7Q10 values are also shown from the baseline for comparison. For both rivers, the mean of BLL median estimates of the 7Q10 across all years is very similar to the mean unconditional estimate, suggesting that the BLL method is relatively free from significant, systematic departures from the baseline. That is, the conditional 7Q10 estimates exhibit a central tendency very near the stable, unconditional 7Q10.

Figure 6.

Median estimates of the 7Q10 for the (a) Susquehanna and (b) Connecticut Rivers developed for each year using the BLL (blue solid) and baseline (red dashed) methods. SDLF observations (black points) and the springtime NAT (dotted line) are also shown. The differences between median 7Q10 estimates predicted by the BLL and baseline methods are also shown for the (c) Susquehanna and (d) Connecticut Rivers (blue dashed line). All differences are expressed as a percentage of the median baseline estimate. The original time series of differences are shown along with their 10 year moving average (black solid).

[48] Despite their similar central tendencies, BLL 7Q10 estimates do exhibit moderate, year-to-year differences from their unconditional counterparts. The series of annual differences and their 10 year moving averages for both rivers are shown in Figures 6c and 6d. All differences are expressed as a percentage of the median, baseline 7Q10 estimate. Differences between median 7Q10 estimates range from –11.3% to +5.3% for the Susquehanna River and from –14.0% to +6.0% for the Connecticut River. In both rivers, a decadal pattern emerges in the differences, as shown by their 10 year moving averages. This suggests the presence of low-frequency variability in the springtime NAT that has propagated through to the estimates of critical low-flow risk.

[49] The most significant interannual differences occur in the first half of the record. Drought conditions lingered in the Northeast during these times, along with a persistent negative phase of the NAT. The signal between the NAT and SDLFs is strong when the NAT is negative (Figures 1c and 1d), with tighter groupings of very low SDLFs during this climate phase. Because of this tight relationship, local neighborhoods defined for years with significantly negative NAT values contain many of the lowest SDLF values in the record, and these values often fall near the middle of the neighborhood region. The kernel then assigns these extreme low flows large weights in the likelihood function, explaining the significant, downward departures from baseline estimates during these times.

[50] The magnitude in differences between BLL and baseline 7Q10 estimates becomes smaller later in the century for both rivers, when the NAT enters a persistent, positive phase. This could potentially be caused by two different factors. First, the prior distributions could be restricting Weibull parameters from taking values that would produce significant, positive departures of the 7Q10 over baseline estimates. In addition, there appears to be greater variability in SDLFs during years exhibiting positive NAT events (see Figures 1c and 1d), even though a general upward trend is present. The increased variability in SDLF values during positive NAT events could lead the BLL method to more evenly weight a wider range of SDLF values in its likelihood function during these times and produce 7Q10 estimates that more closely resemble their unconditional counterparts.

[51] The results of the naïve-prior BLL approach are used to determine whether the asymmetry of BLL 7Q10 estimates around those of the baseline is caused by restrictive priors or the heteroskedasticity in the SDLF-NAT relationship. Figures 7a and 7b show the same differences from baseline estimates as shown in Figures 6c and 6d, but now the differences between 7Q10 estimates developed under the naïve-prior BLL and baseline approaches are also included. While baseline deviations associated with the BLL approach and its naïve-prior counterpart are very similar during the first few decades, the naïve-prior BLL approach exhibits much larger, positive departures from the baseline in the second half of the record. Since the only difference between the two methods is the choice of priors, this result suggests that the regional prior distributions used in the BLL approach are restricting Weibull parameters from adopting values that could produce higher 7Q10 estimates. Upon further investigation, the prior distribution for the scale parameter appears to be constraining for both rivers. The regional prior used in the BLL approach strongly restricts the scale parameter from adopting larger values deemed likely under the naïve-prior BLL, consequently suppressing very large 7Q10 estimates in later decades when the NAT is positive.

Figure 7.

Same as in Figures 6c and 6d, but the differences between median 7Q10 estimates predicted by the baseline and both the BLL (blue dashed) and naïve-prior BLL (green dotted) methods are shown.

[52] An analysis of the entire posterior distribution of 7Q10 estimates, rather than just median values, can provide further insight into the BLL approach by enabling an examination of its estimation uncertainty. Figures 8a and 8b show the width of the 95% credible intervals (i.e., the difference between the 97.5th percentile and the 2.5th percentile) for 7Q10 estimates developed under the unconditional baseline, BLL, and naïve-prior BLL approaches. Wider credible intervals indicate greater estimation uncertainty. Two main results emerge from this figure. First, there is a decadal pattern in the estimation uncertainty of the BLL approach. Estimation uncertainty for the 7Q10 is fairly similar between the BLL and baseline approaches in the first half of the record for both rivers, but after 1970, the width of the credible interval for the BLL method increases significantly. One possible cause of this is the increased variance in the SDLF-NAT relationship during this time (when NAT values are persistently positive). Second, the estimation uncertainty of the BLL approach is consistently and significantly less than that of its naïve-prior counterpart. At times, the width of the 95% credible interval for the naïve-prior BLL approach is over 50% larger than that of the BLL method. This shows that the use of informative priors can significantly reduce the uncertainty in parameter and quantile estimation. This feature of the BLL approach is especially useful because the local likelihood function effectively censors some of the data from the analysis, which increases estimation uncertainty.

Figure 8.

The width of the 95% credible interval (CI) for 7Q10 estimates developed from posterior distributions estimated for each year under the BLL (red dashed), naïve-prior BLL (green dotted), and baseline (blue solid) methods.

[53] Finally, we examine the ability of the BLL approach to provide an accurate assessment of critical low-flow risk. In this application, the risk of low flows is accurately represented, by definition, if SDLFs fall below 7Q10 estimates 10% of the time. In practice, the appropriateness of a given method cannot be rejected if this percentage varies from 10% within what would be expected due to sampling error. This can be evaluated using a binomial test that determines the statistical significance of SDLFs falling above and below the 7Q10 level with respect to their expected frequencies of 90% and 10%, respectively. Table 1 shows the percentage of years that SDLFs for each river fall below median 7Q10 estimates under the baseline, BLL, and naïve-prior BLL approaches. Also included are the p values from the binomial test. The results in Table 1 suggest that the BLL approach may be slightly conservative in estimating low-flow risk, particularly in the Connecticut River, while the naïve-prior BLL appears most consistent with the level of risk associated with the 7Q10. However, given the results of the binomial test, all three approaches sufficiently quantify the risk of critical low flows and cannot be rejected at the 0.05 significance level.

Table 1. Percentage of Years in Which Median Estimates of the 7Q10 Exceed SDLFsa
 Baseline (%/p value)BLL (%/p value)Naïve-Prior BLL (%/p value)
  • a

    P values from the binomial test are also shown.


5. Discussion and Conclusions

[54] Static water quality regulations may no longer be appropriate given nonstationary hydrologic conditions and the implications of climate change. There is a pressing need to develop more robust and sophisticated risk assessment tools in water quality management so that water quality agencies can better protect aquatic environmental quality at low cost under changing hydrologic conditions. A flexible statistical tool that provides such information was presented here. The Bayesian local likelihood approach was able to inform estimates of critical low-flow quantiles with seasonal forecast information and account for estimation uncertainty. The approach provides a mechanism to imbue low-flow risk assessments with new research that can both explain and predict the physical processes that govern low-flow basin dynamics.

[55] The application to two major rivers in the northeast United States demonstrated that the BLL method is stable, robust, and generalizable across space. The semiparametric structure of the approach can flexibly account for nonlinearities and heteroskedasticities in the predictor-predictand relationship, and regional information embedded in prior distributions can stabilize quantile estimates. The use of informative priors can significantly reduce the uncertainty in parameter and quantile estimates, as was seen in the comparison of estimation uncertainty between the BLL and naïve-prior BLL approaches.

[56] Interestingly, the results also suggest that informative priors can restrict parameters to ranges that may be considered too narrow in nonstationary frequency analyses. For instance, the prior on the scale parameter used in the BLL analysis kept 7Q10 estimates relatively low, even during times of low drought risk. The restrictive nature of the scale prior distribution used in this study was a direct result of the static fitting procedure that considered the entire period-of-record for gauges used in the regional analysis. If the regional analysis also incorporated forecast information in the model fitting procedure, then the prior could be more responsive to changes in the NAT signal and allow larger scale values to be adopted during years with a positive NAT event. We believe there is significant opportunity for future research exploring the use of dynamic prior distributions that encapsulate regional predictor-predictand signals in nonstationary frequency analyses. Recent work exploring these ideas in the context of trend detection may provide guidance for future research in this area [Renard et al., 2006; Hanel et al., 2009; Neppel et al., 2011].

[57] More broadly, nonstationary frequency analyses have the potential to improve the efficacy of water quality regulations that currently depend on static probabilistic characterizations of low-flow risk. Under the prevailing regulatory system, entities that release effluent into a receiving water are often mandated to maintain a level of treatment that can prevent the violation of water quality standards under all flow conditions greater than some lower threshold (e.g., the 7Q10). When a static frequency distribution is fit to SDLF data, a few anomalously low values can dominate the model fit, driving 7Q10 estimates very low. Stringent and expensive treatment procedures are then required to meet water quality targets that are more conservative than needed for most years. The economic implications of excessive treatment for local municipalities and businesses, especially those without state support, can be drastic. The implications are also significant for water quality trading markets, in which the total pollutant load for a basin and potential trading incentives that emerge from that pollutant cap are determined using the expected low-flow conditions for that system.

[58] Conditioning the lower streamflow threshold on skillful predictors provides a mechanism to isolate stringent water quality standards to only those years when they are truly needed and relax regulations during years when they impose an unnecessary burden on waste-generating entities. Seasonal forecasts can inform regulatory standards with probabilistic information about the physical mechanisms that drive low-flow processes, allowing regulatory agencies to tighten standards when drought conditions are likely and loosen requirements when rivers are more likely to have higher flows and more dilution power. If seasonal forecasts can explain and predict the worst low-flow conditions, the conditional frequency analysis can isolate particularly drastic low-flow events from the frequency analysis for other, more “normal” years. This will effectively reduce the overall sensitivity of the frequency analysis to particularly low outliers, preventing the development of overly conservative regulatory standards or pollutant caps. Alternatively, in flexible regulatory systems such as water quality trading markets, effluent standards can be made stricter prior to anticipated drought conditions to prevent “die-off” events that may arise from higher pollutant concentrations under very low-flow conditions.

[59] This research focused on developing tools that could condition risk assessments for low flows on seasonal hydrologic predictors, but the approach can be generalized to condition a low-flow frequency analysis on longer-term predictors like climate and land-use change projections. Nonstationary frequency analyses like the BLL method may be critical in adapting the current water quality regulatory framework to shifting hydrologic conditions due to global and regional change.