## 1. Introduction

[2] If a model accurately represents processes relevant to the simulated system, errors in simulated predictions depend on parameter detail that is not represented in the model, and/or is not accurately inferred through calibration. In recognition of this, research has been undertaken to develop approaches for evaluating the potential error associated with model outputs. *Tonkin et al.* [2007] describe two broad methodological approaches as predictive error variance analysis (PEVA), and predictive uncertainty analysis (PUA). Some benefits and shortfalls of each approach are now briefly summarized.

[3] Linear and nonlinear PEVA evaluate the potential error in predictions made by a calibrated model using methods based upon variance propagation [e.g., *Bard*, 1974; *Seber and Wild*, 1989]. Since PEVA evaluates the error in predictions made by a calibrated model, some form of regularization is used to obtain a unique solution to the inverse problem. This is usually a form of parameter parsimony. Therefore, PEVA is usually undertaken using a small number of parameters. Nonlinear PEVA is formulated as a constrained optimization problem in which a single prediction is maximized or minimized subject to the constraints of maintaining the model in a calibrated state at a certain level of confidence [*Vecchia and Cooley*, 1987]. *Tonkin et al.* [2007] present a method for undertaking nonlinear PEVA, based on the method described by *Vecchia and Cooley* [1987] but extended to highly parameterized models in which mathematical regularized inversion methods are used to estimate parameters. In that approach, there is (notionally) no limit to the number of parameters that can be included in the analysis.

[4] Nonlinear PEVA possesses efficiencies that stem from its basis in Gauss-Newton techniques [e.g., *Cooley and Naff*, 1990], and can be effective when calculating confidence intervals for a single prediction. However, nonlinear PEVA must be undertaken independently to investigate different model predictions, so that efficiencies achieved from the analysis of a single prediction are eliminated by the necessity of undertaking as many constrained minimizations/maximizations as there are predictions.

[5] PUA is a more intrinsic concept that explicitly acknowledges that many parameter sets enable the model to reproduce available observations within confidence limits determined by the error associated with measurements of system state and the innate variability of system properties. In principle, no single calibrated model is identified. Instead, a suite of parameter realizations is generated. PUA techniques include calibration-constrained Monte Carlo, Markov Chain Monte Carlo (MCMC), and other methods that propagate prior stochastic parameter descriptions through a model [e.g., *Kitanidis*, 1996; *Yeh et al.*, 1996; *Oliver et al.*, 1997; *Kuczera and Parent*, 1998; *Woodbury and Ulrych*, 2000]; the method of stochastic equations [*Rubin and Dagan*, 1987; *Guadagnini and Neuman*, 1999; *Hernandez et al.*, 2006]; Generalized Likelihood Uncertainty Estimation (GLUE) [*Beven and Binley*, 1992]; and deformation techniques [e.g., *Lavenue and de Marsily*, 2001; *Gómez-Hernandez et al.*, 1997, 2003].

[6] MC-based PUA is appealing since it does not typically rest upon assumptions of model (quasi-) linearity; and, once a suite of parameter realizations has been obtained, this ensemble can be used to evaluate the uncertainty associated with *any* model output. However, the application of MC-based PUA can be complex and computationally intensive when honoring calibration constraints on parameters. In particular, calibration-constrained MC-based PUA can be onerous when forward model run times are long and/or when a large number of parameters are included in the analysis.

[7] In one comparative analysis, *Gallagher and Doherty* [2007] determined that while nonlinear, calibration-constrained PEVA was more efficient than MCMC PUA when examining the range of uncertainty associated with a single model prediction, this relative efficiency diminished when the uncertainty of many predictions was evaluated, and/or where there was a need to examine the statistical relationships between multiple model predictions. This suggests that the desirable benefits of MC-based PUA to evaluate multiple predictions and their statistical relationships could be capitalized upon if the computational efficiency could be improved upon.

[8] In this paper we describe a new subspace Monte Carlo (SSMC) technique that enables efficient evaluation of the range of error associated with outputs from highly parameterized models. The SSMC technique is founded in error variance analysis theory: however, it incorporates several developments that render it akin to deformational MC techniques. In the new SSMC technique, the model is calibrated using a subspace regularization technique such as Truncated Singular Value Decomposition (TSVD) or the hybrid Tikhonov-TSVD superparameter approach. Using sensitivities calculated with the calibrated model, the calibration null-space is defined, which is spanned by parameter combinations that (if the model were linear) have negligible effect on simulated equivalents to available observations [*Tonkin and Doherty*, 2005]. Next, a stochastic parameter generator is used to produce multiple realizations of the model parameters. For each realization, a difference is formed between the stochastic parameters and the calibrated parameters. This difference is then projected onto the calibration null-space, added to the calibrated parameters, and the model executed.

[9] If the model is no longer calibrated, parameter combinations that comprise the calibration solution space, which is orthogonal to the calibration null-space, are reestimated with the null-space-projected parameter differences retained as additive values. This recalibration can often be undertaken using superparameters constructed on the basis of existing sensitivities, so that conditioning may require only a small number of model runs. It is also demonstrated that the SSMC technique enables the inclusion of fine (e.g., model grid) scale parameterization even when the calibration is undertaken using parameterization devices such as zones or pilot points.

[10] This paper is structured as follows. First, the theory of regularized inversion using subspace techniques is presented. Equations are presented that describe (1) how calibration can be formulated as subspace-based regularized inversion, (2) how postcalibration parameter error variance is calculated, (3) the projection of stochastic parameter values onto the calibration null-space, and finally, (4) Broyden's rank-one update [*Broyden*, 1965] and how this can benefit the SSMC technique. Following this, the SSMC analysis procedure is described fully, together with an introduction to methods for incorporating fine (e.g., model grid scale) detail that was not included in the calibration but that honors spatiotemporal parameter statistics. Assumptions that underlie the SSMC technique, and their possible repercussions, are then discussed. The SSMC technique is then demonstrated by application to a synthetic groundwater model, and to a real-world watershed model. These applications demonstrate that the SSMC approach is general (that is, it is not limited to any particular model or to any particular parameterization device) and that it can rapidly produce a large number of conditioned parameter sets for use in assessing the uncertainty in a variety of model outputs. Results of these analyses are discussed together with concluding remarks.

[11] To our knowledge this is the first presentation of a general method for efficiently evaluating the range of model outputs for any model employing any parameterization technique. Fundamental to the method is the use of a large number of parameters, and the use of subspace techniques. Use of a large number of parameters may allow property detail to be represented in a model at a scale approaching true variability of these properties (or at least a scale that approaches that to which predictions of interest are sensitive); at the same time, the use of subspace techniques allows the analysis to be undertaken efficiently as an adjunct to calibration undertaken using regularized inversion.