Risk-based planning offers a robust way to identify strategies that permit adaptive water resources management under climate change. This paper presents a flexible methodology for conducting climate change risk assessments involving reservoir operations. Decision makers can apply this methodology to their systems by selecting future periods and risk metrics relevant to their planning questions and by collectively evaluating system impacts relative to an ensemble of climate projection scenarios (weighted or not). This paper shows multiple applications of this methodology in a case study involving California's Central Valley Project and State Water Project systems. Multiple applications were conducted to show how choices made in conducting the risk assessment, choices known as analytical design decisions, can affect assessed risk. Specifically, risk was reanalyzed for every choice combination of two design decisions: (1) whether to assume climate change will influence flood-control constraints on water supply operations (and how), and (2) whether to weight climate change scenarios (and how). Results show that assessed risk would motivate different planning pathways depending on decision-maker attitudes toward risk (e.g., risk neutral versus risk averse). Results also show that assessed risk at a given risk attitude is sensitive to the analytical design choices listed above, with the choice of whether to adjust flood-control rules under climate change having considerably more influence than the choice on whether to weight climate scenarios.
 Awareness of potential climate change impacts on water systems is becoming well established among western U.S. water managers [e.g., Anderson et al., 2008; Groves et al., 2008]. Regional warming is expected to cause snowpack reductions, more winter flooding, reduced summer flows, increased southwest aridity [Milly et al., 2005; Seager et al., 2007] and greater competition for surface water supplies [Intergovernmental Panel on Climate Change (IPCC), 2007; Bates et al., 2008]. There is also awareness that local impacts will be affected by the rate of warming and potential precipitation changes. Given a range of possibilities, agencies may feel motivated to move beyond scenario-specific impacts investigations and develop adaptive, risk-based planning approaches that robustly portray a spectrum of potential future climates. The Intergovernmental Panel on Climate Change Working Group II reports that some countries and regions have already begun this process, developing adaptation procedures and risk management practices for the water sector on the basis of projected hydrological changes and related uncertainties [IPCC, 2007]. In developing such procedures, managers require information on which of their operational and planning decisions are unlikely to be affected by climate change, which will probably be affected and could benefit from adaptive decision support in the short term, and which will probably be affected but involve adaptive actions that can and probably should be deferred to later dates [Freed and Sussman, 2006]. Ultimately the implementation of such procedures leads to characterization of underlying climate risks, adaptive capacities, system modification options, and interrelations between them in order to suggest management priorities through time.
 The primary contributions of this paper are twofold: (1) demonstrate a new framework for revealing reservoir operations risks under climate change and apply that framework in a California case study; and, (2) reveal how portrayed risk is sensitive not only to scenario definition (e.g., chosen climate projections) but also to analytical design choices subsequent to scenario definition (e.g., how to weight projections, how to conduct scenario-impacts analysis). To date, a scenario-based approach rather than a risk-based approach has typically been used to study climate change implications for water systems in western U.S. regions, including the Columbia-Snake River Basin [e.g., Hamlet and Lettenmaier, 1999; Payne et al., 2004], the Colorado River Basin [e.g., Nash and Gleick, 1991; Christensen and Lettenmaier, 2007], and the California Central Valley [e.g., Lettenmaier and Gan, 1990; Brekke et al., 2004; Anderson et al., 2008; Vicuna and Dracup, 2007, and references listed therein]. Scenario-based approaches have been beneficial because they have built awareness of potential impacts relative to plausible “what-if” scenarios (e.g., a climate projection reflecting choices of greenhouse gas emissions pathway and climate modeling approach). In a risk-based framework, a scenario-based approach is one of several embedded elements (section 2). The objective is to produce probabilistic impacts information on the basis of consideration of a scenario ensemble, analysis of scenario-specific impacts, and on estimates for relative scenario probabilities derived from a frequency-based perspective [Vick, 2002]. The benefit of risk-based planning is that the underlying risk assessment reveals impact magnitudes in a context that simultaneously addresses multiple decision-making attitudes toward risk. For example, median impacts might be more relevant to “risk-neutral” decision makers whereas more extreme and low-probability impacts might be more relevant to “risk-averse” decision makers.
 The risk-based framework described in this paper provides a new context for climate change assessments. It is a flexible framework that advances methods for assessing scenario-impacts and/or characterizing uncertainties about projection scenarios or associated impacts. In fact, the development of this framework was influenced by recent efforts focused on characterizing water resources impacts uncertainty [Maurer and Duffy, 2005; Maurer, 2007; Christensen and Lettenmaier, 2007; Wilby and Harris, 2006]. These studies considered climate projection ensembles rather than a small set of scenarios in order to describe potential impacts in a distributional context. The need to consider climate projection ensembles is evident for California, where there is greater agreement among temperature projections but little consensus among climate models on future precipitation projections [Dettinger, 2005; Cayan et al., 2008; Maurer, 2007]. While there are conceptual models of how northern midlatitude precipitation may respond to global warming [e.g., Seager et al., 2007], it remains unclear how precipitation may respond to global warming in regions such as northern California, which are located between broadening subtropical subsidence zones [Seager et al., 2007] and wetter northern midlatitude zones [IPCC, 2007]. Including an array of model projections of precipitation ensures adequate characterization of uncertainty, since most of the variability among models is due to differences in precipitation projections [Maurer and Duffy, 2005].
 In the efforts discussed above, climate projection ensemble members were treated as equally plausible. Other recent studies have questioned this assumption and offer methods to rationalize unequally weighted climate projections. Such methods typically involve characterizing climate projection distributions [Dettinger, 2005, 2006], which may be weighted on the basis of assessment of relative climate model skill [Tebaldi et al., 2005; Brekke et al., 2008]. Granted, when climate projection distributions are used to infer unequal scenario weights, or likelihoods, the resultant weights are only relative to one another and valid within the limitations of the climate projections evaluation. True estimates of scenario likelihood cannot be generated without characterizing all of the uncertainties associated with climate projection (e.g., those related to global economic and technology developments, resultant emissions pathways, biogeochemical responses, etc.). In this sense, the actual uncertainties of future climate may be greater than that represented by the collective of available climate projections because the latter only represents a limited range of potential climate forcings and responses [e.g., Stainforth et al., 2005]. Nevertheless, it is clear that research efforts will continue to advance our methods for assessing projection uncertainty, scenario likelihoods and scenario impacts; and, as those advancements come to fruition, a risk-based framework for climate change assessment (such as that proposed herein) offers a context where those advancements can be readily transitioned to adaptation decision support.
 The remainder of this paper is outlined to address the two objectives stated above: propose and demonstrate a risk assessment framework, and apply that framework multiple times to reveal sensitivity of portrayed risk to analytical design. Section 2 introduces the framework and its initial application for the case study demonstration. The demonstration focuses on operations risk for California's Central Valley Project (CVP) and State Water Project (SWP) systems (Figure 1). Section 3 then goes on to address the paper's second objective to demonstrate that analytical design decisions made in setting up the risk assessment can affect portrayed risk and possibly affect subsequent decision making. The four designs explored in this case study stem from two analytical design choices:
 1. Whether to assume that contemporary flood control rules will persist under climate change, or that such rules will undergo modification as climate change affects runoff tendencies on storm to seasonal timescales.
 2. Whether various climate change scenarios should be assumed to have relative unequal likelihood on the basis of evaluation of contemporary climate projection information, or equal likelihood.
 These choices and the two defined options for each choice represent a very minor fraction of the analytical designs that might be considered in this analysis, which is discussed further in section 4.5. The purpose of the paper's second objective is not to comprehensively reveal risk sensitivity to the multitude of design possibilities. Instead, the purpose is to illustrate (1) how different designs influence risk portrayal and implicitly influence decision making at a given risk attitude, and (2) how design choices are relatively more or less influential on risk portrayal and thereby may warrant more or less research attention if the goal is to reduce uncertainty about portrayed risk.
2. Methods: Risk Assessment Framework
 The risk assessment methodology includes four primary steps (Figure 2): (1) recognize the planning context and decide on relevant risk assessment metrics and look-ahead periods; (2) survey an ensemble of climate projections to define an array of future climate possibilities and use that context to weight a subset of projections that will be analyzed in detail for impacts, (3) analyze scenario-specific impacts on hydrology and associated operations, and (4) integrate results from the first three steps to construct probabilistic distributions of impacts (i.e., portrayed risk). The following sections introduce the framework and its initial case study application. It is important to understand that this paper defines risk as the range potential change in operations performance relative to baseline operations and future climate possibilities. Doing so permits the reader to understand results without having to be oriented with the case study system's baseline performance and management details. For each risk metric, change is “relative to baseline” and presented as percentage change in metric conditions (sections 4.3 and 4.4).
2.1. Decision Drivers
 The context of a given planning decision determines which look-ahead period(s) and operational performance metric(s) are relevant in a given risk assessment (see decision drivers in Figure 2). Two types of planning decision were conceptualized for the case study: (1) those associated with changes that can be implemented on the shorter term (e.g., use of existing system but with changed “plan of operation”), and (2) those involving commitment to longer-term system modification (e.g., infrastructure developments). Subjectively, two decision application periods (look-ahead periods) were defined as they might relate to these shorter- and longer-term decisions, respectively: 2011–2040 and 2041–2070.
 On choice of risk metrics to describe CVP and SWP operational performances, a large collection of metrics might be used. The relevance of any specific metric depends on planning objective and operations characteristic of concern (e.g., flood protection, water supply reliability, environmental habitat support, hydropower generation potential, and recreational service). In this demonstration, focus was on two metrics that broadly relate to water supply reliability and a variety of planning questions: (1) mean annual water delivery to the CVP and SWP “export service areas” located south of the Sacramento-San Joaquin Delta, and (2) mean end-of-September upstream-of-delta storage at the two primary upstream reservoirs (i.e., Lake Shasta (CVP) and Lake Oroville (SWP)) (Figure 1). Joint consideration of these two metrics highlights the competing objectives of maximizing current-year water delivery versus enhancing drought protection for subsequent years by reserving some stored water for use in those subsequent years (i.e., carryover storage). Subjectively, this study focused on mean annual conditions for each metric during “drier” years only, to emphasize situations where operations are more likely to be supply stressed and climate sensitive, leading to more pronounced competition between these operating objectives. “Drier” years were defined to be either “dry” or “critical” years classified according to the Sacramento Valley Water Year Index [California State Water Resources Control Board (SWRCB), 1994]. The latter index gives weighted consideration to the previous and current year hydrologic conditions in the Sacramento Valley (e.g., between 1921 and 2003 there were 30 years classified as “dry” or “critical”). The decision to assess metric values averaged over “drier years” is only a subjective choice. Alternatively, the risk metrics could have been based on extreme annual or month conditions rather than multiple-year averages, which might be more relevant for questions related to severe flood or acute water shortage possibilities.
2.2. Scenarios and Weights
 The risk scenarios arise from available climate projection information, which collectively represents future possibilities for climate forcings (i.e., emissions pathways) and a host of approaches for simulating future climate response to these forcings (i.e., climate models and how they're applied). Scenario assumptions in this study began with the survey of two different climate projection ensembles. The larger and encapsulating ensemble (see box 2.a in Figure 2) included 75 projections (see uncertainty ensemble in Table 1) sampled from the World Climate Research Programme's (WCRP) Coupled Model Intercomparison Project phase 3 (CMIP3) multimodel data set [Meehl et al., 2007]. The uncertainty ensemble provides a basis for estimating relative likelihoods (see box 2.c in Figure 2) for the members of a subset ensemble (see impacts ensemble in Table 1 and box 2.b in Figure 2), which are to be analyzed in detail for impacts (see scenario-impacts analysis in Figure 2). The decision to develop this framework with encapsulating uncertainty and nested impacts ensembles is based on recognition that many planning situations will have computational capacities that limit the feasible size of an impacts ensemble relative to the available amount of projection information [Brekke et al., 2008]. The impacts ensemble of this case study happens to be the same projection ensemble featured in the hydrologic impacts assessment of Maurer . The rest of this subsection describes how impacts ensemble members were weighted, which involves a process of surveying the uncertainty ensemble, fitting climate projection density functions, and using the relative “climate change coordinates” of impacts ensemble members within those functions to infer relative scenario weight.
Table 1. Climate Projections and Models Included in This Case Study
From information at Lawrence Livermore National Laboratory's Program for Coupled Model Diagnosis and Intercomparison (PCMDI): http://www-pcmdi.llnl.gov/.
Run numbers assigned to model- and scenario-specific IPCC  projections at the PCMDI multimodel data archive and reflect the initial conditions of the given projection.
Impacts ensemble projections initialized from model's 20c3m run 1 for all models except giss er (20c3m run 3 initializing both A2 and B1 projections), ncar pcm1 (20c3m run 2 initializing both A2 and B1 projections, and ukmo hadcm3 (20c3m run 2 initializing the B1 projection).
A2 and B1 represent IPCC  storylines A2 and B1, respectively.
Numbers in parentheses correspond to impacts ensemble scenario order (1–22).
 Membership in the uncertainty ensemble followed Brekke et al. , and was guided by two objectives: (1) include a range of emission pathways that largely represent the range of possibilities considered in IPCC AR4 [IPCC, 2007], and (2) include climate models that had been used to simulate the selected pathways in (1) as well as past climate forcings during the 20th Century Climate Experiment [Covey et al., 2003]. The resultant 75-member ensemble represents 17 of the 23 CMIP3 climate models and their collective projections of both pathways A2 and B1 from the IPCC Special Report on Emissions Scenarios (SRES) [IPCC, 2000], which depict relatively faster and slower rates of atmospheric greenhouse gas accumulation, respectively.
 Projection density functions were next developed to characterize “spread” of information from the uncertainty ensemble. Subjectively, two “climate change” aspects were selected: period change in mean annual surface air temperature (T) and total precipitation (P) near a common location in the study region (i.e., a northern California location near 122N and 40W in this study). These changes were computed for each look-ahead period (2011–2040, 2041–2070) relative to a historical base period (1950–1999). Climate change projection density functions (Figure 3) were then fit to each look-ahead period's pool of values (i.e., 75 values per period) using nonparametric density estimation [Scott, 1992; Wilks, 1995], Gaussian kernel functions with optimized bandwidths [Silverman, 1986], and a product kernel [Scott, 1992] to extend univariate functions for T and P into a bivariate function.
 Finally, to estimate relative scenario likelihoods for impacts ensemble members, the coordinates for each member's paired T and P values are identified within the associated period-specific function (i.e., the numbers in Figure 3 corresponding to member numbers in Table 1). Density values at member coordinates are gathered (and arbitrarily scaled collectively so that they sum to the number of scenarios). The rescaled values are accepted as relative scenario weights where “weight > 1” implies that the Impacts scenario agrees relatively well with broader projection consensus and “weight < 1” implies the contrary (see Figure 4; discussed further in section 4). While this case study does not consider climate model veracity, the framework does permit such consideration. However, Brekke et al.  considered the same uncertainty and impacts ensembles of this case study and showed that reducing the set of 17 contributing climate models to a “better half” set of models, on the basis of apparent model veracity, ultimately had little effect on the estimated weights for the impacts ensemble members.
2.3. Scenario-Impacts Analysis
 Numerous studies have presented methods for assessing climate change impacts on California hydrology and CVP/SWP operations, and this case study follows just one set of those methods. For example, focusing on hydrologic impacts analysis, Vicuna and Dracup  compared 36 published studies on California hydrologic impacts under climate change, mostly focused on the Sierra Nevada and southern Cascade Mountains. The most common approach among these studies involves simulating runoff response to climate change. This requires generating weather sequences at a time step consistent with the chosen hydrologic simulation model (typically daily or 6-hourly) and with the aspects of downscaled climate projections to be represented (e.g., changes in monthly climatology [Miller et al., 2003]; monthly evolving climatic conditions [Maurer, 2007]). Simulated runoff results are then translated into adjusted surface water supplies for operations analysis [e.g., VanRheenan et al., 2004; Anderson et al., 2008].
 For simulating scenario runoff response, this demonstration follows Miller et al. . The approach was applied in nine headwater basins relevant to CVP, SWP, or local district operations (Table 2). The chosen hydrologic model was the Sacramento Soil Moisture Accounting (SacSMA) model [Burnash et al., 1973] coupled to the “Snow17” model on the basis of the Anderson snow model [Anderson, 1973], provided by the National Weather Service (NWS) California Nevada River Forecasting Center (CNRFC). CNRFC staff calibrated and provided basin-specific SacSMA/Snow17 applications (see Table 2 for calibration periods and metrics) and provided calibration period weather forcings (i.e., 6-hourly mean area temperature (mat) and precipitation (map)). Model calibrations were conducted according to NWS procedures [Anderson, 2002] and involved multiple objectives, including matching peak flow rates and monthly volumes (P. Fickenscher, CNRFC, personal communication, 2007). Assessment of model calibrations revealed that simulation biases by flow regime were minimal (results not shown).
Table 2. Headwater Basins Included in the Runoff Impacts Assessment
Labels and descriptions are from the National Weather Service California-Nevada River Forecast Center.
Elevation represents basin area-average above mean sea level.
CNRFC developed hydrologic simulation models. For calibration metrics the first value is for daily volumes; the second value is for monthly volumes. Calibration period is 1 October 1962 to 30 September 1992 for all basin models except that for FRAC1, which was calibrated during 1 October 1965 to 30 September 1993.
Trinity at Claire Engle Reservoir
Sacramento at Delta
San Joaquin at Friant Dam
Tuolumne at Hetch Hetchy Dam
Middle Fork Feather at Merrimac
North Fork Yuba at New Bullards Bar Dam
North Fork American at North Fork Dam
Stanislaus at New Melones Dam
Merced at Pohono Bridge
 Generation of scenario weather sequences followed Miller et al.  and reflected historical-to-future period changes in climate. First, simulated monthly mat and map were computed for each basin scenario combination from the 1950–2099 gridded monthly time series of downscaled climate simulations (see Table 1 and Maurer ). Second, for each of three simulated periods (1963–1992, 2011–2040, and 2041–2070), observed-to-simulated period mean monthly mat differences and map ratios are computed. These differences and ratios are then used to generate new 6-hourly weather sequences by shifting (mat) or scaling (map) the observed 6-hourly mat and map.
 This weather generation approach was applied for each basin, projection, and period combination (i.e., 9 basins × 22 projections × 3 periods). Runoff response to climate change was then computed for each combination, following Anderson et al. , as the ratio change in mean monthly runoff from “base period” (1963–1992) to “future period.” These responses, labeled streamflow perturbation factors, were then used to adjust runoff-related inputs for operations modeling, as discussed in the next section.
2.4. Reservoir Operations
 This study follows methods of Anderson et al. , including preparation of scenario operations analyses and usage of the joint CVP and SWP planning model, CalSim II [Draper et al., 2004]. CalSim II is a monthly time step decision model developed for exploring what-if supply, demand, and constraint scenarios concerning long-term CVP, SWP and local district operations. Focusing on supply assumptions, CalSim II has traditionally been applied to study scenario operations subjected to a single monthly hydrologic sequence consistent with historical observations (e.g., relative spells of drier and wetter years). The sequence reflects (1) monthly hydrologic observations from 1922 to 2003 (i.e., reservoir inflows, local creek flows, valley floor interactions between groundwater and surface water), (2) removal of historical and transient management effects (e.g., irrigation diversions and return patterns), and (3) reintroduction of scenario management effects. Although this case study follows the traditional CalSim II application and portrays “baseline” operations performance relative to a single hydrologic sequence, the framework is flexible and permits portrayal of baseline operations relative to a broader set of annual sequencing possibilities, potentially derived by stochastic modeling relative to instrumental record or dendrochronologic “climates” (see Prairie et al.  and section 4.5).
 Regardless of whether “baseline” supply variability assumptions are reflected through single or multiple hydrologic sequences, the assumed envelope of supply variability must be adjusted in a climate change investigation to reflect projected impacts on natural runoff (section 2.3.1). Following Anderson et al.  and use of CalSim II, scenario-specific adjusted versions of the baseline hydrologic sequence are developed using the monthly streamflow perturbation factors produced from the runoff response analyses (section 2.3.1). Other runoff-related CalSim II inputs are also modified consistent with reservoir inflow changes, including the annual series of forecast seasonal inflow volumes, hydrologic year type classifications, and rules for determining annual delivery targets relative to demands and water supply in the given simulated year. For forecast volumes and year type classification, the approach was to preserve the 20th century relations between these variables and reservoir inflows, leading to shifted variable values consistent with inflow adjustments. For delivery target rules, the simulated annual delivery logic in CalSim II is made cognizant of long-term water delivery reliability using preliminary simulation and adjustment to delivery target rules. This action of preconditioning annual delivery logic for each scenario reflects some implicit operational adaptation to climate change, and is consistent with the approach taken in previous studies using CalSim II [Brekke et al., 2004; Anderson et al., 2008].
 Aggregate CVP and SWP surface water demands were not changed with climate in this case study. This choice was based on the assumption that while climate change may affect decisions on water usage type and efficiency at the field scale within a particular water use district, the aggregate district-wide demands influencing CVP and SWP operations will not necessarily change. Other institutional, regulatory, and operating constraints in CalSim II were kept the same for all scenarios considered in this default version of the analytical design. Section 3 discusses how one operating constraint, monthly flood control assumption, was revisited in a sensitivity analysis on risk.
2.5. Risk Assessment
 The preceding steps produce an “impacts ensemble” of scenarios, weights and impacts for each look-ahead period and risk (impacts) metric. As a final step, this information is consolidated into a portrayal of risk specific to metric and period. First, scenario impacts (section 2.3) are resampled in proportion to estimated scenario weights (section 2.2) to produce an augmented and weight-proportional set of impacts values (for a given metric and look-ahead period). Next, a density function is fit to this augmented set of impacts values (using techniques from section 2.2) and converted into a rank-cumulative distribution function of impacts. The resultant distribution describes breadth and relative rank-probability thresholds of impacts, simultaneously addressing multiple decision-making risk attitudes (e.g., median impacts or a risk-neutral perspective, lower-percentile cumulative impacts for risk-averse perspectives).
3. Methods: Sensitivity of Risk to Analytical Design
Section 2 introduced the risk assessment framework and described its application for a California case study. The latter included many analytical design choices, referred to here as a subjective and “default” analytical design. Switching to the paper's second objective, the methods in this section address how portrayed risk can be sensitive to design choices and options. In this case study, two analytical design choices were permitted to vary, setting up four parallel risk assessments and revealing sensitivity of portrayed risk relative to given risk attitudes (e.g., risk-neutral and sensitivity of “median” impact relative to analytical design). This section describes the two choices and methodology used to establish an alternative design option for choice 1.
3.1. Choice 1: Assumptions about Future Flood Control Constraints
 This choice relates to scenario-impacts analysis (section 2.3.2). In the default design, no operating constraints were adjusted for the scenario climate changes considered, except for the annual delivery targeting rules which were conditioned consistent with scenario-supply statistics (section 2.3.2). Alternatively, flood control constraints on reservoir storage operations might have been modified as climate change might be expected to affect hydrologic event potential relevant to flood control rules. Current flood rules for Shasta and Oroville reservoirs call for an autumn storage drawdown to create “flood space” for controlling potential winter and spring runoff events. Rules then permit “refill” to begin during spring, when necessary “flood space” and risk of flood events decrease on the basis of historical observations. Previous studies have suggested that Central Valley river tributaries will experience increased frequency, magnitude, and duration of winter flood peaks given current climate projections [Dettinger et al., 2004]. Consequently, an important question is how climate change effects on regional flood events would manifest into apparently necessary changes in flood control operations at CVP, SWP and local district reservoirs, and changes in the monthly flood control constraints modeled within CalSim II (section 2.3.2).
 In reality, flood control rules vary relative to runoff potential in the upstream basin, reservoir storage capacity, and downstream channel capacity, and are constrained by Congressional authorizing legislation that dictate how much storage can be seasonally manipulated for flood damage reduction. Such rules are revisited periodically by the regional flood control jurisdiction, and are based on consideration of updated hydrologic, basin, and societal information (E. S. Townsley, U.S. Army Corps of Engineers, personal communication, 29 March 2007). Progressing into the 21st century under a changing climate, these periodic retrospective evaluations could reveal trends in runoff events that control rule specifications (e.g., expectation that a flood-relevant runoff event would occur once in every “X” years). Further, such trends could vary by season. For example, increased rainfall-runoff potential during the winter season owing to warming could lead to greater “flood space” requirements during winter, while earlier spring runoff associated with warmer conditions and reduced snowpack could affect permitted timing of spring refill.
3.2. Choice 2: Estimation of Scenario Weights
 This choice relates to likelihood characterization of impacts scenarios (section 2.2). The default design subjectively assumed that scenarios could be weighted unequally in proportional to the scenario's projected climate change relative to the composite of available projection information (uncertainty ensemble and density functions; see section 2.2). Scenarios featuring a “climate change” closer to the consensus are weighted greater in the default approach. Alternatively, scenarios might have been weighted equally, assuming them to be equally plausible. Weighting affects the final application step of the risk analysis framework (section 2.4) where impacts are resampled in proportion to scenario weights in order to produce a portrayal of risk.
3.3. Sensitivity Analysis
 Consideration of the two design choices required revisiting steps 2–4 of Figure 2.
 1. For choice 1, the options were to assume current or modified flood control, the latter being consistent with climate change impacts on runoff. This required two sets of operations impacts analyses, one for each option (section 2.3.2).
 2. For choice 2, the options were to assume unequal or equal scenario weights. This choice, combined with choice 1, set up four sets of resampled impacts values and risk distributions under step 4 (section 2.4).
 For choice 1, a potential rationale for assuming modified flood control under climate change was vetted with federal, state, and local flood control operators at two workshops hosted by the U.S. Army Corps of Engineers Sacramento District (29 March 2007 and 24 May 2007, Sacramento, CA). Although these discussions did not lead to quantifying climate-dependent runoff drivers of seasonal flood control rules in the Central Valley, they did reveal that regional flood control rules broadly relate to multiday runoff volumes having multidecadal reoccurrence intervals. For illustration purposes in this case study, a flood-relevant runoff-impacts metric was defined as the month- or season-specific maximum 3-day runoff volume during a 30-year runoff simulation, or 3D30Y for a given month or season. In other words, the 3D30Y metric was assumed to be a proxy indicator of seasonal flood control needs and was used to indicate whether climate change might trigger modification to monthly flood control rules. Accordingly, 3D30Y values were identified from each basin-, scenario-, and period-specific runoff simulation described in section 2.3.1. Ratio changes in 3D30Y from base-to-future periods were then computed for the dominant flood control “season” (November–March) and the traditional refill month (April). Subjective review of ratio changes was then used to determine plausible adjustments in November–March and April flood control requirements (section 4.3).
 Admittedly, the use of 3D30Y in this case study reveals only how changes in climatology (i.e., period mean monthly T and P changes reflected in the simulations of section 2.3.1) might translate into adjusted flood control needs. Other potential aspects of climate change are not represented, such as changes in storm types and frequency, and impacts on watershed land cover affecting rainfall-runoff dynamics. Also, any societal changes in flood protection priorities stemming from climate change are not reflected here.
4. Results and Discussion
 This section initially presents results on scenario weights, scenario impacts, and integration of these components into portrayed risk. Discussion then switches to results showing the sensitivity of portrayed risk to analytical design choices 1 and 2 (section 4.4). Finally, limitations on this case study's portrayal of risk are discussed, along with highlighting framework flexibility to consider supply variability not featured in this case study (section 4.5). Results are presented initially for CVP metrics and subsequently summarized for both CVP and SWP metrics.
4.1. Scenarios and Weights
Figure 3 shows climate projection density functions for paired temperature and precipitation change (d(T,P)) over northern California. The functions suggest consensus that some amount of regional warming should occur by both future periods with central warming estimates near 1°C and 2°C during the 2011–2040 and 2041–2070 periods, respectively. The central tendency for P seems to be for relatively little change during either future period with change uncertainty spanning drier to wetter possibilities.
 The function-derived weights for impacts ensemble members (Table 1) are shown in Figure 4. Each member's weight is inversely proportional to the density value at its “climate change coordinate” relative to the density values of other members (section 2.2). The weights given to various impacts scenarios were found to be relatively complex, with no simple set of weights applicable to both look-ahead periods and projected conditions. Results show some scenarios are weighted differently between sequential planning periods (i.e., 2011–2040 versus 2041–2070).
 Review of the impacts ensemble “coordinates” within the uncertainty ensemble functions (Figure 3) suggests that perhaps a different set of impacts ensemble scenarios might have been chosen to provide a more balanced coverage of the uncertainty ensemble's scenario space. As it was, the chosen impacts ensemble features what appear to be several “fringe” scenarios skewed toward relatively warm and wet projections in this study (i.e., scenarios 5, 6, 11, 14, and 17). Note that scenario “climate change coordinates” shown in Figure 3 (and again in Figure 7) only describe projection uncertainty at a common location. Although this was the chosen approach, there's no reason why alternative weighting schemes couldn't have been used that were based on spatially distributed or “mean area” projection uncertainty.
4.2. Runoff Impacts
Figure 5 shows example runoff impacts for one case study basin (i.e., North Fork American River, NFDC1 in Table 2), showing how simulated mean monthly runoff changed from 1963 to 1992 means by scenario, month and look-ahead period. Seasonal tendencies in runoff change were found to be similar among the 9 headwater basins analyzed, and generally involved increased runoff from late autumn through early spring (i.e., wet season), and decreased runoff from mid spring through early summer (i.e., traditional snowmelt and early dry season months). This is consistent with findings from previous studies in the Sierra Nevada [e.g., Miller et al., 2003; Maurer and Duffy, 2005; Maurer, 2007], which identified intervening mechanisms of warming, reduced snowfall and snowpack development, and increased rainfall proportion of precipitation. However, Figure 5 shows that during any specific month, the increment of runoff change varied considerably among the 22 impacts ensemble scenarios considered.
 Switching to daily runoff results and 3D30Y calculations, Figure 6 shows results for the Middle Fork Feather River basin (MRMC1 in Table 2), showing how base-to-future period changes in 3D30Y, distributed across impacts ensemble members, varies depending on look-ahead period and season of 3D30Y occurrence (i.e., 3D30YNov–Mar and 3D30YApr). For MRMC1, there is majority agreement among scenarios that 3D30YNov–Mar volumes will increase suggesting that if 3D30YNov–Mar relates to winter season flood control criteria and if downstream flood protection values are to be preserved during these future periods, then deeper drafting requirements might be required leading to more restrictive flood control constraint on water supply management. In relation to spring refill opportunity, the distributions of changed 3D30YApr in Figure 6 suggest a modest likelihood that 3D30YApr will increase by 2011–2040, a more consensus likelihood for increase by 2041–2070. This suggests that as 3D30YApr indicates spring refill opportunity (i.e., the end of the winter draft period) it is doesn't seem reasonable to anticipate earlier refill opportunity with these climate change scenarios.
Table 3 shows that the results found for MRMC1 (Figure 6) are similar to those found in other basins, and lists the median change in 3D30YNov–Mar and 3D30YApr from the 22 scenario-specific changes computed by basin and look-ahead period. While there is broad consistency in results across basins, it is notable that these median changes vary by basin and look-ahead period. On the latter, relates to period-sampling in the projections, where in this study the impacts ensemble's 2011–2040 precipitation climatologies were, as a whole, relatively “wet” compared to the sampled climatologies during 2041–2070. This explains some of the decrease in scenario median 3D30YNov–Mar from 2011 to 2040 to 2041–2070. On the former, basin attributes like elevation lead to variability among basin-specific results. Elevation interacts with evolving climate change to determine the time-varying transition from more to less November–March snowfall. Initial warming leads to snowfall-to-rainfall transition at lower elevations. Continued warming causes this transition to migrate to higher elevations. Enough warming would eventually extinguish this transition potential, and with that the associated changes in flood-runoff characteristics during winter storm events. Further analysis is required to explore basin-specific potential for warming to trigger snowfall-to-rainfall transition and affect flood runoff potential through time.
Table 3. Effect of Climate Change on Three-Day Runoff Event Potentiala
 Results from Table 3 were used to rationalized flood control rules for alternative analytical design choice 1 (section 3.1). Given tendency for increased 3D30YNov–Mar and decreased 3D30YApr, and given range of magnitudes median changes in Table 3, assumptions were made to portray deeper winter drafting requirement without opportunity for earlier spring refill. For winter drafting requirements, a simple adjustment of 10% more winter drafting was imposed under alternative choice 1, for future periods of operations analysis. Admittedly, basin-unique adjustments to flood control should be anticipated under climate change. The use of this simple adjustment is only meant to illustrate system operational sensitivity to flood control constraints.
4.3. Operations Impacts
Figure 7 shows scenario-specific impacts for CVP risk (impacts) metrics, evaluated during the 2041–2070 look-ahead period and assessed using default analytical design (section 3). SWP impacts and results for both look-ahead periods are summarized in section 4.4. Figure 7 indicates impact values on the basis of plot symbol size and color, and also impacts relative to scenario “climate change coordinates” from Figure 3b. Impacts to mean annual CVP exports during drier years ranged between roughly −25 and +15 percent for the 2041–2070 climate change scenarios. Impacts to mean annual Lake Shasta carryover storage partially mitigate these export impacts, where drier years mean annual carryover changes varied from roughly −45 to +20 percent under the same climate change scenarios. Such results illustrate how reservoir operations partially mitigate the effects of monthly runoff impacts on annual water deliveries by increased reservoir flexing and depletion of carryover storage. They also illustrate that even though there is near consensus among the scenarios that spring snowmelt and reservoir inflow volumes would decrease (Figure 5), reservoirs operations are able to capture and manage enough runoff throughout the calendar year such that there is less certainty whether annual deliveries and carryover storage will increase or decrease. Further, inspection of Figure 7 impacts relative to climate change coordinates suggests: (1) that the transition of negative to positive operational impacts seems to be more sensitive to change in annual precipitation rather than annual temperature, and (2) that at the no-impact threshold, there might be a positive relation between annual precipitation and temperature changes. The latter has the implication that for no net impact on either delivery or storage conditions, there may need to be some incremental increase in precipitation to offset incremental increase in temperature. The former suggests that for the CVP system and its current array of storage resources relative to impacts ensemble climate changes, projected precipitation uncertainty played the dominant role in determining operational impacts uncertainty for the performance metrics considered.
4.4. Operations Risk
Figure 8 shows the integration of estimated scenario weights with scenario impact results (Figure 7) to portray risk for CVP operating metrics for impacts ensemble climate changes by 2041–2070. In Figure 7, scenario-specific impacts values were indicated by marker size and shading. In Figure 8, impacts values are indicated by the vertical axis and are shown plotted versus rank-probability.
 Impacts density functions and rank-cumulative distribution functions were fit relative to both scenario weighting assumptions under analytical design choice 2 (section 3.2, equal weights or unequal weights from Figure 4). This produces two portrayals of risk (cumulative distribution curves) in Figure 8, with black curves corresponding to unequal scenario weights from Figure 4 and the gray curves corresponding to equal weights.
Figure 8 results illustrate how portrayed operations risk can be affected by analytical design choice 2. Equal scenario weighting led to a broader impacts distribution and a greater range of perceived risk (i.e., comparing range of impacts spanning 10 to 90 rank-percentiles from gray curves relative to black curves). This is arguably more relevant to a risk-averse decision maker than a risk-neutral decision maker. In contrast, at the median rank-probability threshold, the impact value changed relatively little compared to impact values at tail probability thresholds.
Figure 9 introduces results from the analytical designs involving modified flood control (section 4.2) and illustrates the relative influence of analytical design choices 1 and 2 on portrayed risk. Line thickness relates to choice 1 (thin is the alternate option involving modified flood control rules), and line color (black or gray) relates to choice 2 as shown in Figure 8. In summary, the assessed risk clearly depends on both analytical design choices. For example, focusing on the 2041–2070 period, the median expected change in Lake Shasta carryover storage varies from −21 to −12 percent among the analytical design options considered (i.e., range of median values from the family of 4 curves shown). The expected change associated with the 0.1 rank-cumulative probability varies from −51 to −42 percent. Inspection of Figure 9 suggests that choice 1 introduces more uncertainty about the median and tail impacts than choice 2. This suggests that choice 1 assumptions for future flood control might warrant greater scrutiny than how to weight scenarios (choice 2) within the options considered.
 Expanding the focus to all SWP and CVP metrics considered in this case study; Table 4 shows the range of impacts from the four portrayals of risk in Figure 9 sampled at three rank-probability thresholds (0.1, 0.5, and 0.9 rank-probabilities). The themes in CVP results are largely repeated in the SWP results. Reservoir operations serve to distribute the effects of monthly runoff impacts to decreases in mean annual deliveries and carryover storage. Likewise, the sensitivity of operations risk for both systems seemed to depend on analytical design choices. Perhaps one notable difference between CVP and SWP risks lie with export results, where expected impacts to SWP exports at the 0.1 and 0.5 rank-probabilities were generally less adverse than corresponding expected impacts to CVP exports, especially under 2041–2070 climates. This result is affected by capacity for upstream-to-export conveyance, for which SWP has greater capacity, and points to potential merit of having greater conveyance flexibility when operating water systems to mitigate climate change impacts on runoff.
Table 4. Sensitivity of CVP and SWP Operations Risk Relative to Multiple Risk Attitudes and Varying Across Multiple Analytical Designsa
From four sets of risk, as portrayed for CVP metrics in Figure 9.
−21 to −15
−9 to −1
4 to 15
−15 to −9
−2 to 4
9 to 20
Lake Shasta carryover storage
−34 to −25
−14 to −3
6 to 21
Lake Oroville carryover storage
−36 to −26
−11 to 0
9 to 27
−30 to −24
−13 to −6
−1 to 13
−27 to −19
−5 to −1
8 to 24
Lake Shasta carryover storage
−51 to −42
−21 to −12
0 to 18
Lake Oroville carryover storage
−54 to −44
−20 to −12
2 to 24
 Case study results should be viewed as risk assessments conditional on surveyed climate projections and analytical assumptions, and with potentially significant uncertainties not quantified and represented. While the risk framework can accept broader consideration of analytical designs and incorporation of uncertainties, there were many key assumptions not explored in the case study, including those related to: climate forcings (e.g., greenhouse gas emission pathways and ultimate translation into perturbed biogeochemical cycles and climate forcing conditions); climate simulation (e.g., physical paradigms and computational limitations); projection downscaling (e.g., how monthly timestep, large-scale climate projections will translate at more local scales and with what submonthly temporal variability); watershed response (e.g., how the chosen surface water hydrologic model represents hydrologic processes differently than other surface water models; how long-term groundwater and/or land cover responses under the scenario climate projections and periods are considered in the analysis; how snowpack reduction through time might erode seasonal water supply predictability and with that the reliability of forecast-driven operations); social response (e.g., how water and energy demands evolved under climate change and affect reservoir operations); and discretionary operational response.
 On assumptions related to social response and operational discretionary response, very few changes were considered in this study. The default analytical design depicts a mostly “static operator” under climate change, with operations strategies kept unchanged except for the annual delivery targeting and the seasonal flood control rules. The “static operator” approach has been featured in numerous climate change impacts assessments on CVP and SWP operations [e.g., Brekke et al., 2004; Anderson et al., 2008]. In contrast, other studies have depicted an “omniscient operator” having perfect foresight of hydrology in coming decades, associated with climate change [e.g., Tanaka et al., 2006]. “Omniscient operators” studies are useful for screening management strategies better suited for climate change conditions, potentially involving market-driven departures from current institutional and regulatory constraints on operations. Both types of simulated operators are unrealistic with the former leading to exaggerated depiction of impacts and the latter suggesting overly optimistic performance driven by perfect foreknowledge of hydrology. However, given that it seems impossible to predict how operations strategies might co-evolve under a changing climate (e.g., reactive or progressive, steadily or in step changes), it might be beneficial to use both representations to future risk studies.
 On the matter of portraying supply variability, it is worth highlighting the limited portrayal of annual sequencing possibilities portrayed in this case study. By following methods outlined in section 2.3.2, results only reflect operations risk relative to potential monthly mean climate and runoff changes superimposed on a historically experienced sequence, and not risk relative to potential changes in climatic variability and annual sequences. The risk assessment framework is flexible enough to give consideration to the latter. Applications of the framework might do well to consider an enriched set of sequence possibilities to more robustly portray risk, especially with respect to management questions involving drought possibilities. For example, recent methods that translate climate projection monthly sequences of temperature and precipitation into monthly sequences of runoff [e.g., Maurer, 2007; Christensen and Lettenmaier, 2007] might be used rather than the chosen method of only representing historical-to-future period mean monthly changes in temperature and precipitation to runoff response (e.g., following Miller et al. ). Additionally, alternative sequence possibilities might be generated using stochastic techniques (e.g., following Prairie et al. ), reflecting hydrologic statistics associated with the “base” or “projected” climates but featuring different “plausible” spells of drought and surplus periods.
 This paper introduces a flexible framework for assessing reservoir operations risk under climate change, and demonstrates application of this framework multiple times in a California case study to show how risk portrayal is sensitive to analytical design choices. Decision makers can apply this framework to their systems by selecting future periods and risk metrics relevant to their planning questions, and collectively evaluating system impacts relative to an ensemble of climate projection scenarios (weighted or not). The case study focused on operations risk for the California CVP and SWP systems, where chosen risk metrics relate to deliveries and carryover storage. The methodology features four primary steps: (1) recognize the planning context and decide on relevant risk assessment metrics and look-ahead periods; (2) survey an ensemble of climate projections to define an array of future climate possibilities and use that context to select and weight a subset of projections that will be analyzed in detail for impacts; (3) analyze scenario-specific impacts on hydrology and associated operations; and (4) integrate results from the first three steps to construct probabilistic distributions of impacts (i.e., portrayed risk).
 Case study results on runoff impacts were found to be consistent with prior studies. Notable impacts included increased winter runoff, reduced spring-summer reservoir inflows, and a net annual reduction in surface water supply that can be controlled through CVP and SWP reservoir operations. Results from operational impacts in the CVP and SWP systems were also consistent with prior studies. Key results included:
 1. Operational partitioning of water supply impacts into decreases in mean annual water deliveries and reservoir carryover storage, with the latter acting to dampen impacts on deliveries.
 2. Mean annual delivery and carryover storage impacts depending more on mean annual P change than on mean annual T change affecting spring-summer runoff.
 3. Smaller decreases in export service area deliveries for the “conveyance richer” SWP system than the CVP system, possibly highlighting the utility of greater conveyance flexibility when managing climate change effects on water supply.
 Scenario weighting and operational impacts results were integrated in risk portrayal (i.e., cumulative impacts distributions) that simultaneously communicate risk to multiple decision-making risk attitudes (e.g., risk neutral and risk averse). Risk results for both systems' delivery and storage metrics were found to be sensitive to both analytical design choices considered. In terms of relative influence, this case study found the scenario weighting decision, as it was framed, to be relatively less crucial than the flood control assumptions embedded in the analytical design. However, this finding might have been different had there been an available weighting basis that produced greater contrasts in weighting hierarchy, or perhaps produced a basis to cull scenarios from consideration.
 Although this application was demonstrated for northern California, its framework can be applied in other settings. Future research aimed at reducing the uncertainty of future precipitation projections would aid planning efforts. Although many factors contribute to the uncertainty associated with future flood potential, the ability to associate future climate with future flood control constraints may be an area where researchers can focus more readily in the present. Future research may explore a more strategic approach for selecting climate projection scenarios for impacts analysis while preserving risk assessment detail. An objective could be to reduce the number of impacts scenarios and associated computational burden. Also, more effort might be spent on revealing risk sensitivity to a broader set of analytical design choices. This would yield two benefits: results would serve as a more robust risk assessment representing both the spaces of climate change scenarios and potential analytical designs; and, indication of which analytical design choices bear relatively more influence on the assessed risk and thereby potentially warrant relatively more attention in climate change research.
 This project was funded by multiple sources: directly by the Bureau of Reclamation Research and Development Office, Reclamation Mid-Pacific Region Office, and U.S. Army Corps of Engineers (USACE) Engineering Research and Development Center; and through in-kind support from USACE Sacramento District, the California Department of Water Resources, U.S. Geological Survey, and Santa Clara University. We thank the staff at Scripps Institution of Oceanography for compiling and processing projection data sets used in these analyses (with funding provided by the California Energy Commission's California Climate Change Center at Scripps). We also thank the staff at the NWS California-Nevada River Forecast Center for providing runoff model tools and simulation support. Finally, we would like to acknowledge CMIP3 climate modeling groups for making their simulations available for analysis, the Program for Climate Model Diagnosis and Intercomparison for collecting and archiving the CMIP3 model output, and the WCRP's Working Group on Coupled Modeling for organizing the model data analysis activity. The WCRP CMIP3 multimodel data set is supported by the Office of Science, U.S. Department of Energy.