## 1. Introduction

[2] Current practice of uncertainty assessment of hydrologic models hypothesizes potential sources of errors, assumes that it obey certain distribution types and nests these distributions within a Bayesian inference framework [*Kavetski et al*., 2006; *Thyer et al*., 2009; *Schoups and Vrugt*, 2010; *Smith et al*., 2010]. Bayesian inference therefore allows simultaneous modeling of uncertainties due to model and measurement errors. These methods are powerful and yield useful insights for improving model structures. A validation of assumptions is generally made by Q-Q plots by mapping observed quantiles to prediction quantiles for a variable of interest [*Thyer et al*., 2009; *Schoups and Vrugt*, 2010]. A Q-Q plot verifies whether the prediction quantiles follow the observed quantiles, thereby assessing the applicability of the model assumptions.

[3] An extension of quantile regression to hydrologic model selection is proposed, which aims to identify a model, for a given model structure, by minimizing a loss function that asymmetrically penalizes the positive and negative residuals. Here a residual is defined as the difference between a prediction and the observed [*Koenker and Basset*, 1978]. The penalty determines the quantile of the observed data at which the model is being estimated. This contrasts likelihood methods that model an entire distribution by assuming a likelihood function. The likelihood methods do not model 1 quantile at a time. It is equivalent to estimating a model such that the prediction of a variable of interest is as close as possible to a desired quantile of its observations. Observed quantiles may exactly be predicted when the model structure contains the truth. Quantile hydrologic model estimation presented here can be seen as equivalent to an inverse approach to Q-Q plot verification. A model is selected to match an observed quantile as closely as possible, instead of using the quantile to judge how well a model (selected independently using another inference method) replicates that quantile. The underlying motivation is to compare two model structures in terms of its deficiencies in representing the underlying processes (“truth”). In contrast to Bayesian approaches to model selection [such as *Kavetski et al*., 2006; *Thyer et al*., 2009; *Schoups and Vrugt*, 2010] where various sources of errors can be explicitly modeled, no assumption on the cumulative distribution of the residuals is made, where a distribution of residuals is due to unknown measurement errors and model structure deficiency.

[4] A Bayesian approach [*Marshall et al*., 2006; *Kavetski et al*., 2006; *Schoups and Vrugt*, 2010] to model selection is limited in certain aspects. The model parameters sampled based on a formal likelihood function is from a posterior parameter distribution only if the underlying processes belong to the model space or that the model space is fully specified [*Davidson and MacKinnon*, 2004, p. 399]. It is only then that the posterior distribution can be assumed to be proportional to the likelihood function based on Bayes rule and hence it is only then that model estimation (selection) can be based on a likelihood function. This is critical in studying model structure deficiency (in the sense of how limited a model structure is in representing the underlying processes). Innovating a complex error model such as in *Schoups and Vrugt* [2010] may ameliorate such concerns in practice. However, a simple model error that cannot be represented by the family of additive skew exponential power distributed [*DiCiccio and Monti*, 2004] error is sufficient to show the limitations of even such complex error models. For example, when the error due to model structure deficiency is correlated with model predictions [*Pande et al*., 2012a], it leads to an effect that is different from the heteroscedasticity effect. Any estimation technique that ignores its presence (dependence of error on model predictions) leads to a biased model estimate [*Heckman*, 2005]. Yet another limitation is that a posterior density is conditional on data, which for small samples can itself be uncertain due to sampling uncertainty [*Pande et al*., 2009]. This though equally holds for the method proposed here.

[5] A Bayesian approach is superior to the proposed method when its assumptions on the error structure hold. This is because the assumptions on the error model structure define the likelihood function, which when valid yield the “true” parameter values of a hydrological model at the likelihood maximum. For example, *Schoups and Vrugt* [2010] assume that the error distribution belongs to a family of additive skew exponential power distribution [*DiCiccio and Monti*, 2004]. The method proposed in the paper makes no assumption on the structure of uncertainty due to underlying processes or measurement errors. This makes it difficult to isolate the uncertainty due to model structure from measurement uncertainty. However, more often than not, the assumptions on error structure (not just distributional but also how the model error enters the assumed error structure) do not hold. It is in this respect, i.e., of not distinguishing between different sources of error, that the presented method is similar in essence to the generalized likelihood uncertainty estimation (GLUE) methodology [*Beven et al*., 2008]. The measurement uncertainty may, however, be isolated from model structure uncertainty by using noise (due to measurement error) adapted data based on measurement error benchmarking studies [*McMillan et al*., 2012].

[6] Thus a motivation behind this paper is to propose a model selection and deficiency assessment approach that is atleast not constrained by the requirement to possess “strong” a priori information about reality [*Vapnik*, 2002, p. 118]. Quantile hydrological model selection and assessment of model structure deficiencies based on it is therefore proposed. Its central idea is that a bias in model estimation by a method that does not assume any error model contains useful information on model structure deficiencies. Further, such an assessment is holistic when it is over the entire range of predictions (such as quantiles of flows with quantiles ranging from 0 to 1) of a model structure. It employs a loss function of *Koenker and Basset* [1978], based on absolute deviations, as an objective function for estimating models that removes the need to identify quantiles of an observed time series.

[7] A deficient model structure constrains how well a quantile of observed variable of interest can be modeled. Different model structures may constrain its prediction of the same quantile in different manner, introducing different bias in predicting observed quantiles over a range of quantile values. The paper demonstrates that quantile model selection incorporates quantile specific bias due to model structure deficiencies in the asymmetric loss function. The loss function thereby allows an ordering of model structures based on their flexibility to model a quantile. Further, model structure deficiencies may induce two quantile predictions of a model structure to cross, yielding a useful diagnosis of structure deficiency. The methodology in the paper thus provides both quantile model predictions for a given model structure and insights into model structure deficiencies for a collection of model structures.

[8] Quantile hydrological model selection is not the same as a standard quantile regression where the underlying model space is of linear functions. Though the standard quantile regression is also a quantile model selection problem, its model space is restricted (since it is linear). Thus, the extension of quantile model selection to a hydrological model space is nontrivial. This is where the need to formally analyze the properties of quantile “hydrological” model selection arises. One property that is crucial is the noncrossing of quantile predictions [*Koenker and Basset*, 1978; *Keyzer and Pande*, 2009]. The conditions under which quantile predictions do not cross therefore need to be made explicit. Its formal treatment is beneficial as it formalizes the notion of model bias due to model structure deficiencies and the conditions reveal that if quantile hydrological predictions cross, it is due to model structure deficiency. It also reveals that bias in predicting observed quantiles due to structure deficiencies is independent of model parameter dimensionality and is time invariant. These are two strong properties that further allow us to compare different structures in terms of its structural deficiencies.

[9] This paper develops the theory of quantile hydrological model selection and deficiency assessment. Its companion paper [*Pande*, 2013] implements the theory in detail and studies cases of a parsimonious dryland model developed for western India [*Pande et al*., 2010, 2011, 2012a], model structures for Guadalupe river basin [*Schoups and Vrugt*, 2010] and validates the performance of quantile model selection and deficiency assessment on French Broad River basin data using a flexible model structure.

[10] The paper is organized as follows. Section 'Methodology' first introduces the methodology, with implementations on a linear regression model, on a simple three-parameter hydrological model with a threshold and two case studies with complex hydrological models as examples. The latter three studies also compare and contrast the approach with Bayesian and point statistics approaches to model selection to elucidate the utility of the approach. A formal analysis of quantile hydrological model selection is then presented in section 'A Formal Analysis of Quantile Model Selection' that expands upon and generalizes the observations made in section 'Methodology'. Section 'Discussion' then discusses the formal results, finally concluding with section 'Conclusions'.