We present an extension of the Reliability Ensemble Averaging, or REA, method [Giorgi and Mearns, 2002] to calculate the probability of regional climate change exceeding given thresholds based on ensembles of different model simulations. The method is applied to a recent set of transient experiments for the A2 and B2 IPCC emission scenarios with 9 different atmosphere-ocean General Circulation Models (AOGCMs). Probabilities of surface air temperature and precipitation change are calculated for 10 regions of subcontinental scale spanning a range of latitudes and climatic settings. The results obtained from the REA method are compared with those obtained with a simpler but conceptually similar approach [Räisänen and Palmer, 2001]. It is shown that the REA method can provide a simple and flexible tool to estimate probabilities of regional climate change from ensembles of model simulations for use in risk and cost assessment studies.
 Projections of climatic changes at the regional scale are of fundamental importance for an assessment of the impacts of climate change on human and natural systems. The most powerful tools today available to generate climate change information are coupled atmosphere-ocean General Circulation Models (AOGCMs). The regional information obtained from AOGCMs can be further refined through the use of different “regionalization techniques” [Giorgi and Mearns, 1991], however to date the use of these techniques to generate actual projections of climate change has been limited. Hence, in this paper we apply our analysis to the results from a set of AOGCM climate change experiments.
 A number of AOGCMs have been used to simulate the evolution of climate from pre-industrial times to the end of the 21st century using historical observed and future scenarios of anthropogenic and natural forcings [Cubasch et al., 2001]. These models have shown a substantial spread in the simulation of climate changes both globally [Cubasch et al., 2001] and regionally [Giorgi and Francisco, 2000; Giorgi and Mearns, 2002]. This adds a strong element of uncertainty to the prediction of regional climatic changes.
 Because different AOGCMs exhibit varying levels of performance over different regions and for different climatic variables, it is extremely difficult to identify the most reliable model simulations. As a consequence, projections of climatic changes and assessment of related uncertainties are best based on the combined information provided by the ensemble of different AOGCM simulations.
 Several recent studies addressed the issue of producing climate change information from ensembles of AOGCM simulations. Räisänen and Palmer  (hereinafter referred to as RP01) proposed a procedure for estimating probabilities of climate change exceeding given thresholds from ensembles of AOGCM experiments. In their method, this probability is measured by the fraction of the total number of models that simulate a change exceeding the threshold. Kharin and Zwiers  assessed various methods of combining information from different AOGCMs. Their application to the hindcast of the AMIP (Atmospheric Model Intercomparison Project) period (1979–1988) showed that the model ensemble average provided the most skillful results in the tropics and the regression-improved ensemble mean in the extratropics.
 In a previous paper [Giorgi and Mearns, 2002] (hereinafter referred to as GM02) we introduced the Reliability Ensemble Averaging (REA) method. This method allows the calculation of best estimate, uncertainty range and reliability of regional climate change projections based on ensembles of different AOGCM simulations. However, for many applications, in particular to studies of cost and risk assessment, it is useful to provide probabilities of climate change. Therefore, here we present an extension of the REA method to the calculation of the probability of regional climate change exceeding given thresholds. For illustrative purposes we apply this method to the same set of AOGCM transient climate change simulations examined by GM02 and compare the results with those obtained from the method of RP01.
 The REA method is extensively discussed by GM02 and only a brief summary of its salient features is given here. Given N simulations of climate change with different AOGCMs for a certain emission scenario, the REA average (or best estimate) change for a variable (e.g., temperature T) is defined as
Equation (1) is a weighted average of the temperature changes simulated by each model (ΔTi), with the weights given by the normalized reliability factors Ri. These factors are designed to provide a measure of how well a given model reproduces present day climate (“the model performance” criterion) and the degree to which a model-simulated change is an outlier compared to the other models (“the model convergence” criterion). Different measures of the model performance and convergence criteria can be devised. GM02 express Ri in terms of the bias of the model control run with respect to observations and the distance of an individual model-simulated change from the REA average change (equation 4 of GM02). The value of Ri increases as the model bias and distance decrease. In other words, models that are poor performers or outliers are weighted less in the REA averaging so that the most reliable information from each model dominates. Based on the definition given in equation (1), GM02 then calculate the uncertainty limits around the REA best estimate (equations 5 and 6 of GM02) and the reliability of the REA best estimate (equation 7 of GM02).
 The concept of reliability factor can be used to estimate the probability of future climate change from the model ensemble. Before doing that, we note that the likelihood that a given model-simulated change will actually happen is generally not known, since future conditions are not known and therefore the model cannot be validated in its ability to forecast climate change. Also not known is the probability distribution of the simulated changes, since this would require a very large sample of model simulations. As a result, some assumptions need to be made concerning the likelihood of a model outcome. For example, RP01 assume that all model-simulated changes are equally likely.
 In our method, the likelihood associated with a model-simulated change (Pmi) is proportional to the reliability parameter defined by GM02. The normalization of this likelihood yields the definition
In other words we assume that the change simulated by a more reliable model (in the sense defined by GM02) is more likely to occur. From equation (2) it follows that, for a given emission scenario, the probability of a climate change exceeding a certain threshold ΔTth is given by
If all the Ri are equal to one, equation (3) corresponds to the approach of RP01. If simulations for multiple emission scenario are available, and if the likelihood associated with a certain scenario k is indicated with Psk then
where is the probability of the temperature change being greater than the threshold ΔTth when all the scenarios are considered.
 We here apply equation (4) to the set of AOGCM simulations used by GM02. This includes transient experiments for the period 1860–2100 with nine different AOGCMs (models from the CCC, CCSR, CSIRO, GFDL, MPI, MRI, NCAR, NCAR-DOE and UKMO) for the A2 and B2 emission scenarios of Cubasch et al.  (see GM02 for more details). In our study, the present day, or control, climate covers the period 1961–1990, while the future climate covers the period 2070–2099. Following GM02, surface air temperature and precipitation changes for December–January–February (DJF) and June–July–August (JJA) are calculated as the difference between the averages for the future and control periods. The averages are calculated for the 21 regions of Giorgi and Francisco  that were also used by GM02. For brevity, we only present results from a subset of 10 regions spanning a variety of latitudes and climatic settings.
Figure 1 shows the calculated probability of temperature and precipitation change exceeding a given threshold as a function of the threshold (see equation 4). Data for both the A2 and B2 scenarios are combined in the calculations, and for simplicity the two scenarios are assumed to be equally likely, that is k = 1, 2 and Psk = 0.5 in equation (4). The reliability factors Ri are calculated following equation (4) of GM02.
 All models calculate a positive temperature change in either scenario and over all regions, therefore the probability of warming is equal to 1. As the temperature change threshold increases, the probability decreases until it reaches 0 above a maximum value. This implies that no model projects a greater change than this value in each of the scenarios. This maximum change is of the order of 4–5 K in low latitude regions and 8–9 K in high latitude regions, where the warming is maximum in response to the sea-ice and snow albedo feedback mechanism (e.g., GM02).
 Having a greater sample size, the curves of Figure 1 could in principle be differentiated to obtain empirical probability density functions (PDFs). The S-shape of the curves implies a central maximum surrounded by tails. The width of the distribution is measured by the gradient of the curves in Figure 1, a steeper gradient implying a narrower PDF. It is evident that this gradient varies substantially across regions, being for example maximum (narrow PDF) over the Southern Australia (SAU) region in both seasons and the Southern Asia (SAS) and Western Africa (WAF) regions in JJA. This implies that over these regions the changes simulated by the different models tend to converge more. Examples of relatively lower gradients (wider PDFs) are Central North America (CNA) in both seasons, East Asia (EAS) in DJF and Northern Europe (NEU) in JJA.
 The probability curves for precipitation have the same shape as for temperature, although they exhibit a greater inter-regional spread, especially in JJA. Some of the curves appear to indicate multimodal PDFs (e.g., Mediterranean, MED, and Southern Africa, SAF, in JJA). Note that the precipitation changes are both, negative and positive. For positive precipitation changes, the value in the curves of Figures 1c and 1d gives the probability of a positive change being greater than a given threshold. For negative precipitation changes, the probability of a negative change of magnitude in excess of the indicated threshold is given by 1 minus the value in the curve. A prevalence of probability of positive change is observed. However, a number of noticeable cases show high probability of negative change (e.g., Southern Asia, SAS, in DJF, Mediterranean, Southern Africa and Central North America in JJA). Especially in DJF some regions show not negligible probabilities of positive precipitation change being greater than 20%.
Table 1 shows for the 10 regions analyzed the probability of temperature and precipitation changes being greater than selected thresholds as calculated using the REA method and using the method of RP01 (i.e., assuming a value of 1 for all model reliability factors). It is evident that, although the general probability trends calculated with the two methods are similar, substantial differences are found in a number of cases, both for temperature and precipitaton. This is because the models are weighted differently in the calculation of the probability.
Table 1. Probability of Temperature (upper table) and Precipitation (lower table) Change Exceeding Given Thresholds Using the REA Method and the Räisänen and Palmer  Method (in parentheses)
ΔT ≥ 2K
ΔT ≥ 4K
ΔT ≥ 6K
ΔT ≥ 2K
ΔT ≥ 4K
ΔT ≥ 6K
ΔP ≤ −10%
ΔP ≥ 0.
ΔP ≥ 10%
ΔP ≤ −10%
ΔP ≥ 0.
ΔP ≥ 10%
 The probability of a temperature change being greater than 2 K is generally high, mostly greater than 0.7 in both DJF and JJA. As the temperature threshold increases to 4 K the probability mostly falls in the range of 0.1 to 0.5, while only a few of the regions, the high latitude ones, show a significant probability of temperature change in excess of 6 K. For precipitation, three regions show a probability greater than 0.5 of a decrease in excess of −10% in JJA (Mediterranean, Southern Africa and Central North America). A precipitation increase is most likely over the majority of regions in DJF, while the probability of a precipitation decrease is more prominent over a number of regions in JJA. Note a probability of 1 for precipitation increase in the summer monsoon regions of South Asia and East Asia.
 Note that the use of the model convergence criterion tends to narrow of the PDF of the simulated changes due to the decreased weighting of outlier changes. In order to test the sensitivity of the probabilities to this criterion, we recalculated the values in Table (1) by setting to 1 the component of the reliability factor that measures the convergence criterion, thereby effectively removing it (see GM02). In most cases this resulted in an increase of the probability of change exceeding the highest thresholds. Examples of large increases in the probability of temperature changes in excess of 6 K were Western North America in DJF (from 0.11 to.19), Eastern Asia in DJF (from .23 to .33) and Central North America in JJA (from .12 to .17). Examples of large increases in the probability of precipitation changes being greater than 10% were Western North America in DJF (from .1 to .14) and the Mediterranean in JJA (from .05 to .11). The probability of precipitation decrease exceeding 10% increased from .11 to .16 in Southern Australia in DJF and from .19 to .27 in Northern Europe in JJA.
4. Summary and Conclusions
 In this paper we have presented an extension of the REA method of GM02 to the calculation of the probability of change in climatic variables being above given thresholds. The method employs ensembles of simulations with different models and has been applied to the set of AOGCM transient climate change experiments used by GM02.
 Our approach is conceptually similar to that of RP01 with the difference that instead of assuming that all simulations are equally likely we assume that the likelihood of a simulation depends on its reliability as measured by the reliability factor of GM02. We feel that this is a better assumption than that of a uniform likelihood across models.
 The conceptual framework of the REA-based probability estimation is independent of the specific functional form of the reliability parameter Ri. As discussed in GM02, different types of criteria, functions, variables or processes can be used to measure the reliability of a model simulation. The REA method provides a simple and flexible tool to quantitatively estimate climate changes and related uncertainty and reliability, as well as the probability of change. The method can be applied directly to the model output without the use of intermediate statistical models and it can be used with any type of model, variable or climate statistics. REA methods can thus provide useful information needed to assess the risks and costs associated with regional climatic changes.