Corresponding author: J. P. Mattern, Department of Mathematics and Statistics, Dalhousie University, PO Box 15000, Halifax, NS B3H 4R2, Canada. (firstname.lastname@example.org)
 Numerical ocean models are becoming increasingly important tools for marine research and for management of marine resources. It is therefore crucial that uncertainty in model predictions and model sensitivity to errors in the model inputs be quantified. We performed a combined sensitivity and uncertainty analysis for a realistic physical-biological model of the Texas-Louisiana shelf in the northern Gulf of Mexico. The model simulates the major physical and biological processes involved in the formation of the hypoxic zone that develops on the shelf every summer. With the help of a statistical emulator technique, we introduced uncertainty in selected model inputs and assessed the effects of these uncertainties on the predicted development and spatial distribution of bottom hypoxia. The uncertain inputs we examined belong to two categories: (i) biological inputs including river nutrient concentration, phytoplankton growth rate and initial and boundary conditions of biological variables, and (ii) physical inputs including freshwater river runoff, wind forcing, and mixing coefficients. We show that uncertainty in different inputs has distinct effects on model output which vary in magnitude, time, and space. Uncertainty in physical inputs was found to have a strong impact on estimates of hypoxia, e.g., hypoxic area estimates vary by more than 40%, due to a 20% variation in the freshwater river runoff.
If you can't find a tool you're looking for, please click the link at the top of the page to "Go to old article view". Alternatively, view our Knowledge Base articles for additional help. Your feedback is important to us, so please let us know if you have comments or ideas for improvement.
 Quantification of uncertainty is an important part of numerical modeling. Knowledge of model uncertainty allows for an assessment of the reliability and precision of the model and therefore its general usefulness as a tool for prediction and analysis [Karniadakis and Glimm, 2006]. Model uncertainty is the model error due to incomplete knowledge of the simulated system, e.g., unknown boundary or initial conditions due to errors in the model formulation and equations or due to lack of (computational) resources to simulate the degree of complexity of the system, e.g., a low model resolution. Inherently connected to the concept of model uncertainty is that of model sensitivity which characterizes the response of model output to changes in its input. A model is said to be sensitive to a particular input if a slight change in the input triggers a large change in the output. Sensitivity thus contributes to uncertainty in outputs. Typically, many model inputs are not well known, which, combined with model sensitivity, can lead to large uncertainty in model outputs.
 In this study, we performed a combined sensitivity and uncertainty analysis for a three-dimensional (three spatial dimensions in addition to the time dimension) physical-biological model set on the Texas-Louisiana shelf. The model includes a nitrogen-based biological module coupled with dissolved oxygen dynamics and is described in Fennel et al. [2011, 2013]. In Fennelet al. , it is shown that the model reproduces observed hypoxic extent well for certain configurations. The goal of this study is to assess the sensitivity of these predicted hypoxia estimates to model inputs of two categories: (i) biological inputs, including one biological parameter (the maximum growth rate of phytoplankton), biological boundary and initial conditions, and river nutrient concentration and (ii) physical inputs, including two parameters controlling subgrid scale horizontal mixing, wind forcing, which can have a significant impact on the extent of hypoxia [Forrestet al. 2011; Fenget al. 2012], and the amount of freshwater discharge.
 Previous studies of model uncertainty and sensitivity have focused on physical ocean models [Lermusiaux, 2006; Kimet al. 2010; Thackeret al. 2012]. Sensitivity analyses are also commonly found in the field of ecosystem modelling [Clancyet al. 2010; Makler-Picket al. 2011; Gibson and Spitz, 2011; Melbourne-Thomaset al. 2011] but not typically for coupled three-dimensional biological-physical models. Only a few studies have investigated uncertainty propagation in physical-biological ocean models, e.g., Béalet al.  assessed the effect of mixing errors on biological properties in a physical-biological model, and Cossariniet al.  simulate the error dynamics of a model of the Lagoon of Venice ecosystem using a data assimilative approach.
 Unless uncertainty is directly integrated into the model [see, e.g., Lermusiaux, 2006], an uncertainty or sensitivity analysis entails many model simulations and is thus computationally demanding, in particular for three-dimensional models. Typically, uncertainty analyses represent input uncertainty via random samples using Monte Carlo techniques [Clancy et al., 2010; Kim et al., 2010; Melbourne-Thomas et al., 2011], a computationally inefficient approach that is not practical for three-dimensional models. Emulator-based approaches, such as the polynomial chaos expansion [introduced by Wiener, 1938] which we use here, offer a computationally more efficient alternative for propagating uncertainty in model inputs to their outputs [Xiu and Karniadakis, 2003; Shenet al. 2010; Thackeret al. 2012]. These techniques sample the input probability distribution in a nonrandom fashion and interpolate model output in between samples.
 Our results highlight the relative importance of the biological and physical model inputs on hypoxia predictions. The inputs cause distinct temporal and spatial distributions in model uncertainty, where high uncertainties are typically found on the inner shelf region in summer. We show that model predictions of oxygen concentration, hypoxia, and surface chlorophyll are sensitive to uncertainty in various model inputs, especially the physical inputs which perturb the model's stochastic flow field. Uncertainties in the inputs show strong local effects, such as the oxygen concentration in a specific grid cell, as well as larger scale features, such as the size of the predicted hypoxic area.
2.1 The Emulator: Polynomial Chaos Expansion
 Emulators provide a statistical approximation of unknown model output based on existing output (here, a statistical approximation means an approximation that provides error estimates, see O'Hagan ). Based on the model output approximation, emulators can be used to propagate uncertainty in model inputs (model parameters, initial or boundary conditions, physical forcing, etc.) to its outputs [Xiu and Karniadakis, 2003; Le Maîtreet al. 2004]. For this purpose, emulators use output from existing model simulations that are obtained by varying uncertain inputs. By interpolating between the existing output values, emulators can estimate model output for new, unknown values of the inputs and even approximate the probability distribution of the model output. Polynomial chaos expansion, the technique we use in this study, is an emulator based on polynomial interpolation (approximation error estimates are not provided directly but can be derived from convergence properties, see Thackeret al. ); it has been used previously with ocean models by Thackeret al.  and Matternet al. . In Matternet al. , the approximation feature of polynomial chaos is used to estimate parameter values for a biological model. In this study, we take advantage of the fact that polynomial chaos allows one to make approximations of model output probability distributions and use it to perform an uncertainty analysis.
 An introduction to the polynomial chaos technique is given in Thackeret al.  and Matternet al. . Here, we briefly review the main aspects. In the following, we consider the stochastic model input θ, which can be, for example, a model parameter or even a scaling factor for the model's boundary condition. Because we only consider variations of single model inputs in this study, we assume in the following that θ is one-dimensional. Although we do not consider the multidimensional case here, the theory translates in a straightforward manner into two or more dimensions for multiple inputs [Xiu and Karniadakis, 2002].
 Here, the model output of interest is represented by the function f(x,t,θ). The output may be dependent on space x, time t, and the uncertain input θ. With the polynomial chaos technique, f is approximated by a basis function expansion as follows:
where ak(x,t) are the expansion coefficients, independent of the uncertain input θ, and the kth basis function ϕk(θ) is a polynomial of order k in the parameter space defined by θ. The parameter kmax is the maximum order of polynomials used and determines the accuracy of the approximation, and ϵtrunc is the truncation error. Without cutoff, i.e., for kmax = ∞ , the approximation is exact and ϵtrunc(θ) = 0. However, the number of required model runs grows with kmax, so that computational constraints force us to use relatively small values in applications with computationally expensive models.
 The choice of polynomials in equation (1) is dependent on the probability density function (pdf) of the parameter θ which we denote p(θ). The polynomials are chosen to be orthogonal with respect to p, and all common distributions have specific sets of polynomial basis functions [Xiu and Karniadakis, 2002]. For the beta distribution, which is used here for its finite support, the corresponding set of orthogonal polynomials are the Jacobi polynomials, and ϕk is the kth Jacobi polynomial.
 In order to approximate f in equation (1), the coefficients ak need to be computed. For this purpose, equation (1) is rearranged to express the coefficients ak in terms of f (x,t,θ(i)), the model output at each so-called quadrature point θ(i) for (see Xiu and Karniadakis  for details). This is where existing model output (kmax + 1 model simulations) are required for the emulator-based approximation. Once the coefficients have been computed, two important properties of the output distribution, the expected value and variance of f, are straightforward to calculate. They are given by
Here, is a normalization factor specific to the kth polynomial and independent of θ; S is the support of p (the region where p(θ) > 0). The properties in equations (2) and (3) are the mean and variance of the probability distribution of f based on the distribution of θ. Different inputs will produce different distributions in f; therefore, mean and variance are dependent on θ. Note that other properties of the probability distribution of f, including its pdf can also be approximated with the polynomial chaos expansion (section 2.6).
 When the polynomial chaos-based approximation is expanded beyond a single (one-dimensional) input, the computational cost increases exponentially with the number of stochastic inputs. As a result, a joint analysis of nine inputs with polynomials of order kmax would require (kmax + 1)9 model runs, while only (kmax + 1) × 9 would be required for an individual analysis of each input. Although methods have been proposed to reduce the number of required simulations by performing sparse sampling [see, e.g., Blatman and Sundret, 2010], we are specifically interested in the effect of individual inputs on the model under similar conditions. For this reason, we sample each input distribution in the same manner but restrict the uncertainty analysis in this study to one parameter at a time, i.e., we estimate the univariate distributions of the inputs and not their joint, multivariate distribution.
2.2 Hypoxia on the Texas-Louisiana Shelf
 The Mississippi-Atchafalaya river system is fed by a vast 3,220,000 km 2 drainage basin and supplies a high load of nutrients [recent estimates are at around 1.25 Tg nitrogen per year, Aulenbach et al., 2007] to the Texas-Louisiana shelf. The nutrients fuel phytoplankton growth on the shelf and contribute to the formation of zones of low-dissolved oxygen in bottom waters [Rabalaiset al. 2002; Dagg and Breed, 2003]. Oxygen is depleted when sinking organic material is remineralized by bacteria-consuming oxygen in the process. In summer, when the water is highly stratified limiting oxygen resupply to the bottom waters, zones of low oxygen form. These zones are considered hypoxic when oxygen concentrations fall below a critical threshold of 63 mmol O2 m − 3 (the equivalent of 2 mg L − 1) that is considered harmful to many marine organisms. In fall, when water column stratification is eroded by cooling and increasing winds, the hypoxic area begins to shrink and disappears before reforming the following year. Hypoxia on the Texas-Louisiana shelf has been subject to many studies; a recent overview is given by Bianchiet al. .
2.3 Model Description
 We are using the physical-biological model described in Fennel et al. [2011, 2013], its domain is shown in Figure 1. The model is based on the Regional Ocean Modeling System [Haidvogel et al., 2008] configured for the Texas-Louisiana shelf as described in Hetland and DiMarco [2008, 2012] and uses the biological model component of Fennelet al.  with an additional variable for dissolved oxygen [Fennelet al. 2013]. The main objective of the model is to simulate the biological nitrogen cycle and the distribution of dissolved oxygen over the Texas-Louisiana shelf which is heavily influenced by freshwater discharge and nutrient input from the Mississippi and Atchafalaya rivers [Bianchi et al., 2010; Laurent et al., 2012]. In addition to dissolved oxygen, the biological model component contains two variables for dissolved inorganic nitrogen (nitrate and ammonium), one variable to represent phytoplankton, one to represent zooplankton, and two detritus variables (small, suspended, and large, fast-sinking detritus). The model also contains a chlorophyll variable, allowing for the simulation of photoacclimation, adjustments to the phytoplankton's chlorophyll content.
 In the following, we provide a short overview with a focus on two model configurations for sediment oxygen consumption and the model inputs important for this study. For more detailed information of the model, see Fennelet al. .
2.3.1 Instant Remineralization (IR) and Sediment Oxygen Consumption (SOC) Model Configurations
 As shown in Fennelet al. , oxygen consumption by the sediments is a significant oxygen sink contributing to the development of hypoxia in our study region. Since the choice of parametrization for this process can have a large impact on hypoxia predictions, we use two different parametrizations here: instantaneous remineralization (IR), and the parametrization of sediment oxygen consumption (SOC) by Hetland and DiMarco . In the IR configuration, as the name suggests, sinking organic material is remineralized instantaneously when it reaches the sediments, leading to a consumption of oxygen in the bottom waters above. This approach has the advantage of making oxygen consumption dependent on the amount of organic matter that reaches the sediment, yet has the disadvantage of speeding up the process of remineralization which in reality leads to a slower but longer lasting consumption of oxygen. In the SOC configuration, sediment oxygen consumption and nutrient production are parametrized based on observations and dependent on temperature and water oxygen concentration only. Consumption of oxygen in the sediments is assumed to occur only in water depths shallower than 50 m in the SOC configuration. This approach is able to simulate a more steady, slow consumption of oxygen; yet the oxygen flux is independent of the flux of organic matter to the sediments and thus decoupled from the supply of nutrients. In Fennelet al. , it is shown that both treatments produce reasonable estimates of the hypoxic zone; thus, we examine both the IR and the SOC model configurations and test their response to uncertainty in the inputs.
2.4 Model Inputs of Interest
 The model inputs to which we introduce uncertainty are listed in Table 1; all variations were introduced as simple scalings. For example, uncertainty is introduced into the spatially and temporally varying wind field by multiplying it with a scaling factor, a scalar random variable, described in section 2.4.1 below. The univariate distribution of the scaling factor can be approximated at much lower computational cost than a multidimensional distribution which would be required to represent temporal or spatial variations in the input uncertainty. An alternative approach that would allow for the variation of multidimensional inputs would be a dimension reduction, e.g., via principal component analysis and subsequent variation of the lower dimensional representation [Thackeret al. 2012].
Table 1. The Model Inputs that are Varied in our Experiments
Variation in wind speed over the entire model domain.
 We selected these specific model inputs because they represent some of the least known inputs that we suspect would have a noticeable effect on the biological model output. We decided to include both physical and biological model inputs to compare their relative effects. The phytoplankton maximum growth rate was selected as the only representative of the biological parameters because we wanted to include a wider variety of inputs.
2.4.1 Input Distributions
 Since the aim of this study is to quantify and compare the effect of uncertainty in various model inputs, we applied the same relative amount of variation to each of the inputs. That is, we used a distribution with a constant ratio of standard deviation/mean for all inputs. While these input uncertainties do not reflect our best knowledge of the real uncertainty of the inputs (although accurate uncertainty estimates would be hard to obtain for most of them), this approach allows for straightforward, objective comparison of the effect on the output uncertainty and is analogous in its approach to sensitivity analyses.
 As mentioned above, all variations in the input are based on the variation of a scaling factor that is multiplied with the input quantity, e.g., the amount of river discharge. For a scaling factor that is equal to one, we obtain the baseline model simulation with standard parameters, which represents the scenario we consider most likely. Departure of the scaling factor from one creates perturbations in the model, relative to the baseline results.
 For our experiments, we assumed a beta distribution with a scaled support that allows for values of the scaling factor between zero and two. The scaled beta distribution has the probability density function
Here, α and β are the positive parameters that determine the shape of p, and is the beta function which acts as a normalizing constant. We only consider beta distributions with parameters α = β > 1 which results in symmetric distributions with mean values of one and with variances that can be adjusted via α and β. For this study, we select a standard deviation of 0.2; the resulting distribution is similar to a normal distribution with equal mean and standard deviation (Figure 2). One advantage of the beta distribution is that it is truncated and thus does not allow for scaling factors less than zero. The beta distribution also comes with its associated set of orthogonal polynomials, the Jacobi polynomials [Xiu and Karniadakis, 2003].
2.5 Shannon Entropy as a Measure of Uncertainty for Model Hypoxia
 A parcel of water is defined as hypoxic if its dissolved oxygen concentration is below the critical threshold of 63 mmol O2 m − 3. In the model, the hypoxic area comprises all grid cells with an oxygen concentration below the threshold. If we consider a single grid cell at a fixed point in time, then it can be in one of two states: hypoxic or not hypoxic. When uncertainty is introduced into the model, each grid cell has a certain probability of being hypoxic. The aim is to identify a suitable measure to quantify the uncertainty of the state of the grid cell. If the probability of being hypoxic is zero, then it is certain that the grid cell is not hypoxic, and the measure of uncertainty should be zero to indicate that there is no uncertainty. For the same reason, the measure of uncertainty should be zero if it is certain that hypoxia is occurring and the probability of being hypoxic is one. In all other cases, our uncertainty measure should be greater than zero and should reach its maximum when both hypoxia and no hypoxia have equal probability, i.e., when the probability of hypoxia is .
 The Shannon entropy introduced in Shannon  (originally published in 1948) is a measure of uncertainty that fulfils the above criteria and is defined for a discrete random variable X that has n possible values x1,x2, … ,xn as
Here, pi = Prob(X = xi) is the probability of X taking the value xi, and b is the base of the logarithm and can be chosen based on the application. If any of the pi = 0, then the product pi logb(pi) = 0 as well.
 In this application, X is the state of a grid cell which can take one of n = 2 values: x1 = “hypoxia” and x2 = “no hypoxia”. We let phypox = p1 be the probability of X being hypoxic and choose b = 2. Then, the Shannon entropy simplifies to
The above expression is shown in Figure 3a as a function of phypox and behaves as desired: If the probability of hypoxia is zero (phypox = 0) or one (phypox = 1), then the Shannon entropy is zero. In between, the Shannon entropy is greater than zero and at reaches its maximum value of one. It has been shown generally that the Shannon entropy fulfils the criteria expected from a measure of uncertainty for a discrete random variable [Shannon, 2001; Jaynes, 1957]. In this study, we will use its binary form, equation (6), and refer to it simply as entropy. We will also frequently use the term uncertainty instead of entropy when referring to the property of the model hypoxia estimates.
2.6 Estimating Properties of the Output Distribution
 After performing the required model simulations (performed for the input values at the quadrature points; see section 2.1, Figure 2a), we obtained the model outputs at the quadrature points. These outputs represent samples from the probability distribution of the model output (for a graphical example, see Figure 2). Our goal is to estimate properties of the output distribution from these samples. The polynomial chaos expansion offers a straightforward and numerically efficient way to estimate the mean and standard deviation of the output distribution (equations (2) and (3)). However, other properties we are interested in, such as the median, other quantiles, and the entropy, cannot be obtained in such an efficient way. Instead, we have to resort to a procedure of interpolation and binning for their approximation.
 In the following, we consider a scalar model output f(θ) that is dependent on the univariate model input θ; θ could, for example, be a model parameter. The procedure below describes the approximation of pf(f(θ)), the pdf of the model output, based on p(θ), the pdf of θ. For higher dimensional model output, the procedure needs to be repeated for every entry in the output vector or matrix.
 Create a fine equidistant grid in parameter space, covering values of θ where p(θ) is greater than zero (in practice, greater than a small positive number). Let for i = 1,2, … ,n denote the resulting n grid points. For this application, we select n = 1000.
 Interpolate the model output in parameter space, i.e., estimate based on known output at the quadrature points.
 Form pairs of and p for each grid value, i.e., for i = 1,2, … ,n.
 Bin the values of for i = 1,2, … ,n into m bins (nonoverlapping intervals). Let xj for j = 1,2, … ,m + 1 be the limits of the bins. Based on the number of grid points n, we select m = 100 (see notes below).
 Sum the probability values associated with each bin, i.e., compute
 Finally, normalize the sums to obtain , which is an approximation of the probability of the model output being in the jth bin, i.e., . At this point, we have associated model output values with probability values. After the normalization in step (vi), we have obtained an estimated pdf for the output. This pdf can then be used to obtain the desired distribution properties. In practice, we used m = 100 bins of equal size to cover the interval between minimum and maximum value of the interpolated model output (step (iv)) in conjunction with n = 1000 grid points (step (i)). An increase in the number of bins typically requires a finer grid with more points.
 For the purpose of estimating the entropy of a binary event (such as model oxygen below the hypoxic threshold; see section 2.5), it is sufficient to use two bins in step (iv) so that all output values below the threshold are collected in the first bin. After the normalization step (vi), the sum of input pdf values in the first bin is an estimate of the probability of the event having the first of two possible outcomes (in our example, an oxygen concentration below the threshold).
 We tested the above procedure by estimating the mean and standard deviation of multiple distributions of different model outputs and comparing the values with the estimates obtained directly from the polynomial chaos expansion. These estimates differ by at most 0.005% in our tests and thus agree to the precision that is relevant to this study. Note that the above procedure can be replaced by a Monte Carlo procedure based on random samples obtained from the emulator approximation. However, we found the Monte Carlo procedure to be less efficient as it required a larger sample size and thus more computing time. No matter which of the aforementioned procedures is used to turn the model output into estimates of the model distribution, the limiting factor for their accuracy is typically the number of quadrature points. More terms in the polynomial chaos expansion in combination with more quadrature points lead to a smaller error in the interpolation and more accurate estimates, but they also require more model simulations (section 2.1, Xiu and Karniadakis ).
2.7 Uncertainty in Emulator Estimates
 The emulator approximation of model output contains the truncation or interpolation error ϵtrunc(θ) (equation (1)). To estimate the magnitude of this error for the biological output and the derived uncertainty estimates (section 2.6), we compared the emulator estimates for two different numbers of quadrature points, i.e., for two different values of kmax. Based on the experiments in Matternet al.  with seven quadrature points, we selected kmax = 6 and 8 which corresponds to emulator approximations based on seven and nine quadrature points, respectively. For both values of kmax, we obtained the emulator approximations for most model inputs (section 2.4). (We did not obtain approximations for the nutrient initial conditions and nutrient boundary conditions inputs based on nine quadrature points. These inputs have such a low effect on the output (see, e.g., results in Figure 6) that we feel confident that seven quadrature points are sufficient for the emulator approximation.)
 If the model is well-approximated by the emulator, then the change from seven to nine quadrature points should be accompanied by a small reduction in the truncation error and little change in the output estimates. Output estimates that vary considerably indicate that the output is not approximated well by the emulator with seven quadrature points (a large change in ϵtrunc(θ) would indicate that the sum in equation (1) is not close to converging and thus even nine quadrature points are not sufficient).
 By comparing selected uncertainty estimates that are based on the emulator approximations for seven and nine quadrature points, we found that the truncation error varied considerably among different inputs. Based on the magnitude of the error, we can divide the inputs into two distinct classes. The first class consists of inputs that induce changes into the biological model directly and do not affect the physical model (e.g., biological parameters, biological initial, and boundary conditions). The effect of these inputs is well-approximated by the emulator, and the seven and nine point-based uncertainty estimates are very similar (Figure 4). The second class of inputs consists of those that change the physics of the model and thereby indirectly alter the biology (e.g., both river runoff scenarios, diffusivity, and wind). The biological model output appears to be very sensitive to changes in these parameters, and the emulator-based approximation is not precise at different locations and varies greatly between seven and nine quadrature points (Figure 4). High uncertainties in the emulator estimates are constrained to regions along the coast and are especially evident close to the location of the Mississippi Delta and Atchafalaya Bay. Here, the increase in the number of quadrature points from seven to nine can lead to a notable increase in the estimated probability of hypoxia in one grid cell, accompanied by a notable decrease in a neighboring cell. Our results in section 3 show that these regions of elevated emulator uncertainty are highly dynamic and also exhibit the highest model uncertainty, a finding that is discussed in section 4.
2.7.1 Propagation of Uncertainty into Entropy Estimates
 In light of the high uncertainty for some of the emulator-based estimates of the probability of hypoxia (Figure 4), it is important to note how the emulator's truncation error propagates from probability estimates to entropy estimates (section 2.5). Due to the log transformation in equation (6), the error in entropy estimates is increased for probability estimates close to zero or close to one (corresponding to low entropy values; see Figure 3), whereas entropy errors are diminished at probability values close to 0.5 (corresponding to high entropy values). Low entropy values therefore have a higher uncertainty than high entropy values.
 Uncertainty propagates through the model equations and leads to distinct expressions of uncertainty in the different model outputs. Because they are of high importance, we focus our analysis on bottom oxygen and bottom hypoxia. In order to also gain an understanding of the expression of uncertainty at the surface, we close this section with an examination of uncertainty in the model's surface chlorophyll predictions.
3.1 Baseline Oxygen Dynamics Without Uncertainty
 First, we briefly describe the simulated oxygen dynamics without introduced uncertainty for the SOC and the IR configurations. We focus the description on the temporal development of dissolved oxygen at a location on the shelf between the Mississippi Delta and the mouth of Atchafalaya Bay (from here on simply referred to as the station, see star in Figure 5a) and a spatial snapshot of bottom oxygen across the model domain on 17 July 2004, a time when the simulated hypoxic zone typically reaches one of its largest extents (Figure 5).
 Bottom water oxygen concentration at the station shows a decline throughout the spring of 2004, beginning in April. The decline is briefly interrupted by several mixing events that deliver oxygen-rich surface water to the bottom. Both SOC and IR models show hypoxic conditions at the sea floor in late May and strongly hypoxic conditions throughout most of July and the first half of August. In mid-August, strong mixing increases bottom oxygen and leads to nonhypoxic conditions. At the station location, the IR configuration exhibits more persistent hypoxic conditions with lower oxygen levels in comparison to the SOC model which switches more frequently between hypoxic and nonhypoxic conditions.
 Hypoxic areas extend along the coast downstream of the Mississippi Delta and Atchafalaya Bay in both model configurations (Figures 5a and 5b). In the SOC configuration, the hypoxic area appears less homogeneous and more fractured in comparison to the IR configuration, and the SOC bottom oxygen concentration field shows more small scale variability. For a comparison with observations, see Fennelet al. .
3.2 Uncertainty in Bottom Oxygen
 We now describe the effect of uncertainty in model inputs on model outputs (section 2.4), focusing first on bottom oxygen concentrations at the station. The probability distribution of bottom oxygen and its seasonal development is characterized by quantiles which we averaged over periods of 3 months (Figure 6). (Note that, in comparison to quantiles of the time-averaged distribution, i.e., calculating the time-average first, then the quantiles, these time-averaged quantiles are influenced less by days of extremely high or low oxygen concentrations. They therefore provide a better representation of an average day within each 3 month period.) Based on the interquartile range (the difference between the 0.75 and 0.25 quantiles), which characterizes the dispersion of the output distribution, we found that the effect of the different inputs varies considerably, and their effect shows a strong seasonal dependence.
 Uncertainties in the wind forcing, the main driver of vertical mixing, have the strongest effect on bottom oxygen for both model configurations. Effects of similar magnitude are caused by the horizontal mixing coefficients (hdiff, hvisc; these abbreviations for the inputs are defined in Table 1) and the freshwater river runoff (included in both river and disch). In contrast, nutrient initial and boundary conditions have little to no impact on the bottom oxygen concentration at the station. The phytoplankton growth rate (μ0) and the river nutrient input (river nut) are the only inputs that have a distinctly different effect on the two model configurations. As expected, their effect is relatively strong in the IR configuration because sediment oxygen consumption is determined by sinking organic material, in contrast to the SOC configuration where oxygen consumption is largely decoupled from the biological nutrient cycle. Some of the probability distributions in Figure 6 exhibit considerable skewness and a deviation of the median from the baseline (based on the model without uncertainty; dotted blue line in Figure 6), which are due to nonlinearities in the model.
 The time dependence of the effect of the uncertain inputs is consistent across inputs for both model configurations. From April through September, the effect of the inputs on bottom oxygen is considerably higher than during the rest of the year. The reason for the low effect early and late in the year is that temperatures are lower and the ocean is well-mixed, a model behavior that cannot be disrupted by the uncertainty in the input we prescribed.
3.3 Entropy of Hypoxia
3.3.1 Temporal Development of Entropy
 The entropy of hypoxia quantifies the uncertainty in model hypoxia estimates and reaches its highest value if the probability of hypoxia is 0.5 (section 2.5). The entropy is low for most of the year and near the ocean surface (Figure 7). In these cases, hypoxia uncertainty is small due to a low probability of hypoxia; variation of the model inputs has a very low probability of causing hypoxia near the surface or when the water column is well mixed in fall and winter. High entropy values and therefore high uncertainty are only found close to the bottom and from mid-May to mid-September, in the times and places where hypoxia is likely to occur. Here, the SOC and IR model configurations show some differences.
 In the SOC configuration, the inputs that have a strong effect on bottom oxygen (see previous section 3.1) cause high uncertainties in bottom hypoxia from June through September. The maximum likelihood region of hypoxia, comprised of the model's bottom grid cell where the probability of hypoxia is greater than 0.5 (region outlined in green in Figure 7), varies strongly among different model inputs (compare Figures 7a– 7d) and is characterized by high entropy values. Thus, estimates of the number and duration of hypoxic events that the station experienced in 2004 are highly uncertain. This, of course, is only true for the model inputs that have a strong effect on bottom oxygen, but not for those with low effects such as the nutrient initial conditions (the effect of variations in the initial nutrients for the IR configuration is included in Figure 7k; in the SOC configuration, entropy values are equally low).
 The IR model configuration results in stronger, more persistent hypoxic events at the station (Figure 5d) which correspond to larger maximum likelihood regions (Figure 7e– 7k). Some inputs, such as the river runoff (Figure 7f), exhibit low entropy values within the maximum likelihood region; these correspond to points where the model is nearly certain that hypoxia will occur.
 To summarize the temporal development of entropy, we obtained 3 month averages of bottom hypoxia entropy at the station (Figure 8). In both model configurations, uncertainties due to variation of inputs are close to zero early in the year and become significantly larger in April. During the second half of the year, uncertainties are larger in the SOC configuration compared to the IR, with the exception of those caused by the μ0 and river nut inputs which are generally lower in the SOC configuration due to the weaker coupling between the nitrogen cycle and sediment remineralization (the biological model component affects the sediment oxygen consumption in the SOC configuration through the bottom oxygen concentration only). Generally, uncertainty values of bottom hypoxia correspond well to the variations caused in the bottom oxygen concentrations (Figure 7).
3.3.2 Spatial Distribution of Entropy
 The spatial distribution of hypoxia entropy in mid-July (Figure 9) corresponds well to our previous observations: Outer shelf regions exhibit low uncertainty and a low probability of hypoxia; high entropy values occur on the inner shelf and mostly along the fringe of the maximum likelihood hypoxic area. The maximum likelihood hypoxic area contains low entropy values; these correspond to high probabilities of hypoxia. The station (blue star in Figure 9) is in the fringe of the high entropy region which explains the high uncertainty values in the temporal development observed previously (Figure 7).
 Consistent with hypoxia predictions in the baseline simulations, the SOC model configuration shows a less homogeneous and more fractured region of high entropy compared to IR (compare SOC Figures 9a– 9d to IR Figures 9e– 9i). All model inputs create a region of low entropy within the maximum likelihood hypoxic area close to the Mississippi Delta and mouth of Atchafalaya Bay. These correspond to a high probability of hypoxia and low uncertainty.
 Based on the spatial maps of hypoxia probability, we computed bounds for the size of the hypoxic area on 17 July 2004 (Figure 10). For this purpose, we summed the areas of the model grid cells with a probability of hypoxia greater than a given threshold. For a threshold of 0.5, we obtained the maximum likelihood area of hypoxia. Other thresholds can define uncertainty estimates for the size of the hypoxic area. For example, the interval in between the thresholds of 0.6 and 0.4 corresponds to the size estimate of the area with entropy values ≥ 0.97 (Figure 3a). The wind input, both river inputs and the horizontal mixing coefficients induce wide ranges into the size estimates, especially in the SOC model configuration. For example, for the river inputs river and disch, the area between the 0.6 and 0.4 probability thresholds is greater than 5000 km 2, more than 40% of the maximum likelihood estimate (Figure 10a). In the SOC configuration, the sizes of the maximum likelihood areas for the high uncertainty inputs also deviate strongly from the estimates of the baseline simulation without uncertainty. Here, it is most apparent that the symmetric distribution of the uncertain input can lead to a highly skewed response in the model.
 In summary, some inputs result in large uncertainty in hypoxia predictions, while two inputs (nutrient initial and boundary conditions) have nearly no effect on bottom oxygen. Uncertainty in hypoxia predictions is temporally and spatially constrained to when and where hypoxia occurs, e.g., variation of model inputs cannot induce hypoxic conditions when the water column is well mixed.
3.4 Comparison of Hypoxic Area Estimates with Observations
 For the freshwater discharge, one of the inputs the hypoxia predictions are most sensitive to, we derived a multiyear time series of hypoxic extent (Figure 11). The time series exhibits an annual cycle, with hypoxia typically disappearing from November to March and reaching its maximum extent in late July or August. The uncertainty in the simulated hypoxic extent displays a seasonal cycle, as well, with high uncertainty during the summer months and small uncertainty at times of small hypoxic area estimates. The deviation of the median size of the hypoxic area from the baseline simulation, which was already observed in Figure 10, persists throughout the four years of simulation.
 Using the time series, we perform a qualitative comparison with hypoxic area estimates obtained from observations from the LUMCON cruises [Rabalaiset al. 2007] for the years 2004–2007 (black bars in Figure 11). In the years 2004 and 2006, both the maximum likelihood estimate and the baseline simulation estimate agree with the observation-based estimate (Figure 11a and 11c). In 2005, the baseline simulation is close to the observations, while the simulation with uncertainty yields lower estimates of the hypoxic area so that the observations lie outside of the region with hypoxia probability of 0.25 or higher (Figure 11b). In 2007, the simulation with uncertainty again underestimates the size of the hypoxic area, yet the observations are within the region of hypoxia probability greater than 0.25 (Figure 11d). In other words, by extending the maximum likelihood hypoxic area to the area that has a probability of hypoxia of 0.25 or higher (based on the uncertainty in the freshwater discharge), the prediction corresponds to the observation-based estimate. Uncertainties in the freshwater discharge could help explain the model misfit here.
 It should be noted that the observational estimates contain an error as well due to nonsynopticity in the observations, sampling error in the spatially and temporally varying field, and the error associated with interpolation. Nevertheless, simple analyses like this can help to gain an understanding of model error sources and their effect on model output.
3.5 Uncertainty in Surface Chlorophyll Estimates
 In order to assess the expression of uncertainty in model variables not directly related to oxygen, we briefly focus on the surface chlorophyll estimates of the model. Surface chlorophyll is often used in data assimilation and model validation applications due to the wealth of available satellite observations [see, e.g., Allen et al., 2007; Mattern et al., 2010]. To obtain an overview of the uncertainty in surface chlorophyll, we computed its time-averaged 3 month quantiles at the station (Figure 12). In comparison to the bottom oxygen quantiles (Figure 6), it is evident that the same inputs have a strong effect on both quantities, albeit in a different order. The inputs that include freshwater river runoff (river, disch) have a very pronounced impact on surface chlorophyll at the station, and the extra nutrient input of the river compared to the disch input manifests itself in a slightly more spread out distribution. As with bottom oxygen, seasonal changes in the effect of the inputs are apparent, and all of the inputs have a higher impact in the middle of the year. The choice of SOC or IR model configuration has no noticeable impact on the surface chlorophyll field at the station and leads to negligible differences in the time-averaged quantiles (compare Figure 12a and 12b).
 To get an idea of the spatial spread of uncertainty, we obtained the mean and standard deviation of the surface chlorophyll field on 17 July 2004 for two inputs: disch, which has a strong effect on surface chlorophyll at the station, and μ0 with a weaker effect. Both inputs show similar patterns in the mean surface chlorophyll field (Figure 13). Surface chlorophyll is highest close to the Mississippi Delta and the mouth of Atchafalaya Bay, in regions where the nutrient input into the model is high. As expected, uncertainty, described here by the standard deviation, is lower for the μ0 input. For the disch input, high values of the standard deviation are found along the fringes of the high chlorophyll areas, similar to the spread of entropy along the boundary of the hypoxic area (Figure 9). Variations in the inputs appear to affect the surface patch of high chlorophyll in a similar way they affect the hypoxic zone at the bottom: Spatial offsets in large chlorophyll gradients have a similarly strong effect on model uncertainty as offsets of the boundary of the hypoxic area.
 We performed a series of experiments to assess how uncertainty in selected inputs of a physical-biological model of the Texas-Louisiana shelf propagates into its output. This uncertainty analysis was based on an emulator approach, the polynomial chaos expansion, to approximate output distributions based on prescribed input distributions. We selected nine inputs from both the physical and biological components of the model and used the same relative amount of uncertainty for all inputs. We focused on uncertainty in estimates of oxygen and bottom water hypoxia expressed through the biological model component. The goals of this study were to examine the effect of different inputs in time and space and to compare their impacts. Two benthic parametrizations, IR and SOC, were considered in the examination. While our experiments were performed using the polynomial chaos expansion, we expect that similar results can be obtained using other emulator approaches or simpler random sampling-based Monte Carlo techniques, albeit the latter typically require a higher computational effort in the form of more model simulations to reach a similar level of accuracy.
 In order to quantify the uncertainty in model estimates of hypoxia, we used entropy, a simple function of the probability of hypoxia (Figure 3a). In cases where we are interested in finite numbers of events (here, there are two events: the oxygen at a grid cell concentration can either be below the hypoxic threshold or above), entropy is the suitable way to quantify uncertainty [Shannon, 2001]. High entropy values that we encounter on many occasions (Figures 7 or 9) indicate that model hypoxia estimates can be highly uncertain.
 As we have shown in section 2.7.1, the emulator-based uncertainty estimates themselves are not free of error. While uncertainties in the biological inputs (river nutrient load, phytoplankton maximum growth rate, nutrient initial, and boundary conditions) are propagated into the output reliably, there is a higher error when estimating the model response to variations in the physical inputs (wind, freshwater discharge, and vertical mixing coefficients; Figure 4). Lower emulation error can be achieved by adding more terms to the polynomial chaos expansion; yet this requires more quadrature points, i.e., more samples from the input distribution, and comes at a much higher computational cost. What is interesting in regard to the emulator error is that it spatially correlates with the region of high model uncertainty (compare Figure 4b and Figure 9). The cause of the error is that the emulator does not sample the output on a sufficiently dense grid to capture its variability. One reason for this model behavior is that, in contrast to the biological inputs, changes in the physical inputs can affect the stochastic flow field, e.g., instabilities and eddies in the model, leading to highly variable small scale changes. It is this high sensitivity of the model to the physical inputs that leads to the emulator error. For this reason, we hypothesize that the high model uncertainty estimates are not caused by a systematic overestimation by the emulator, but rather that both model uncertainty and emulator error are caused by the same phenomenon, the highly variable model output due to perturbations in the circulation. Thus, the emulator estimate is right at least qualitatively by assigning high uncertainty values to the regions in question.
 The results of this study offer insights into the sensitivity of the model and allow for an assessment of the impact of the uncertain inputs at each point in time and space of the model simulations. The most obvious result is that two inputs, the nutrient boundary and initial conditions, have very little influence on the simulated bottom oxygen concentrations and the surface chlorophyll content, while the model is considerably more sensitive to the other inputs we investigated. Uncertainty in physical model inputs had the strongest effect on both bottom oxygen and surface chlorophyll. This includes freshwater river runoff, consistent with other studies identifying rivers as large sources of uncertainty in coastal regions [Blumberg and Georgas, 2008; Cossariniet al. 2009]. While changes in the nutrient boundary conditions had little effect, our model also has a high sensitivity to the physical boundary conditions, as shown in Fennelet al. . Variations in input to the biological model, including the river nutrient load, had generally a much smaller impact.
 Besides model sensitivity, this study provides some insights into model uncertainty, due to errors in the inputs, although we cannot provide a full assessment of model uncertainty here, since the model responses to inputs were observed individually and not all inputs were considered. Most importantly, we deliberately chose the same relative error for all inputs in order to better compare their effect, instead of selecting a specific distribution for each input which would reflect our actual knowledge of the input processes. However, we can draw some important conclusions with regard to model error from our results. Uncertainty in the inputs typically has little effect on oxygen or chlorophyll outside the biologically very productive zone close to the river mouths or outside the spring and summer months (April through September). Within the region and period of higher error, however, uncertainties due to just a single input can be considerable. Estimates of hypoxia are strongly influenced by all of the physical inputs we investigated (Figure 9), as are size estimates of the hypoxic region (Figure 10). Even if the input uncertainties are decreased (in this study, we used a distribution with an standard deviation of 20%), it is very likely that their combined impact on bottom oxygen estimates should lead to high errors in hypoxia estimates.
 In section 3.4, we contrasted hypoxic area estimates based on observations with uncertainty estimates due to river inflow. We have shown that the variations we introduced into the river inflow can, in three out of four cases, explain the discrepancy with the observations. Assuming that the error in the observations is relatively small, the predicted uncertainty cannot explain the discrepancy that occurs in 1 year. Here, an adjustment of the river inflow in the model would thus likely not lead to much improved fit to the observations, and one has to turn to other sources of model uncertainty for improvement.
 We analyzed the impact of uncertainty in nine physical and biological model inputs on hypoxia predictions from a physical-biological model of the Texas-Louisiana shelf in the northern Gulf of Mexico. For the purpose of propagating uncertainty from model inputs to the outputs, we used a model emulator, the polynomial chaos expansion, which offers a straightforward approach to propagating uncertainty. While the computational cost of an emulator-based uncertainty analysis is significant, it offers the possibility of a thorough assessment of model sensitivity for a select number of model inputs.
 We found that the impact of the different inputs varied considerably: the physical inputs typically had a stronger effect than the biological inputs, with variations in the river freshwater discharge and the wind field having the strongest impact on both concentrations of bottom oxygen and surface chlorophyll. Two of the biological inputs we evaluated, nutrient initial and boundary conditions, had a negligible effect on the output we assessed. Strong effects were constrained to regions with high biological activity in both time and space. The uncertainty introduced into the system is considerable and especially affects estimates of hypoxia and the hypoxic area. For example, hypoxic area estimates of the model vary by more than 5000 km 2 or by around 40% (probability of hypoxia between 0.4 and 0.6) due to 20% variation in the river inflow.
 We thank Peter Jan van Leeuwen, Bruce Smith, and Keith Thompson for valuable comments on an earlier version of this manuscript. We also thank three anonymous reviewers whose comments led to improvements. This work was supported by NOAA CSCOR grants NA06N0S4780198 and NA09N0S4780208 and the U.S. IOOS Coastal Ocean Modeling Testbed. NOAA NGOMEX publication no. 174.