Temperature dependent climate projection deficiencies in CMIP5 models



[1] Monthly mean temperatures for 34 GCMs available from the CMIP5 project are compared with observations from CRU for 26 different land regions covering all major land areas in the world for the period 1961–2000 by means of quantile-quantile (q-q) diagrams. A warm period positive temperature dependent bias is identified for many of the models within many of the chosen climate regions. However, the exact temperature dependence varies considerably between the models. We analyse the role of this difference as a contributing factor for some models to project stronger regional warming than others by looking at the entire ensemble rather than individual models. RCP4.5 temperature projections from all GCMs for two time periods (2021–2050 and 2071–2100) are compared against a linear fit to the 50% warmest months from the respective q-q plot for each model and region. Taken together, we find that in general models with a positive temperature dependent bias tend to have a large projected temperature change, and these tendencies increase with increasing global warming level. We argue that this appears to be linked with the ability of models to capture complex feedbacks accurately. In particular land-surface atmosphere interactions are treated differently and with different degree of realism between models.

1. Introduction

[2] An increase in global mean annual temperature with best estimates ranging from 1.8 –4.0 °C is projected towards the end of the century for the B1 and A1FI IPCC SRES scenario, respectively [Intergovernmental Panel on Climate Change (IPCC), 2007]. How this global mean value is to be realised at a regional level is the subject of intensive research activities worldwide. The main tool for constructing regional climate projections are global climate models (GCMs) and it has long been realised that an ensemble approach is necessary in order to be able to sample uncertainty in some manner [Annan and Hargreaves, 2011]. It is also known that this uncertainty is a combination of various sources out of which some are reducible (e.g. by model improvements), while others are inherent to the problem [e.g., Mastrandrea et al., 2011; Refsgaard et al., 2012]. One aspect of reducing the uncertainty by model improvement not easy to assess for an individual model is related to how it behaves under yet unrealised climate conditions. The fundamental assumption therefore is that by constructing climate models from first principles taking as many identified physical interactions of importance into account as possible, the model will be robust enough to be able to simulate perturbed climate with credibility. Taking a multi-model ensemble approach would then frame uncertainty assuming more or less random deviations from the most likely development by the individual models. Thus, it is central to all modeling efforts to minimize systematic errors.

[3] Results from detailed analysis of an ensemble of regional climate models (RCMs) within the European project ENSEMBLES [Van der Linden and Mitchell, 2009] clearly suggested that biases tend to influence the interpretation of anthropogenic climate change signals. Christensen et al. [2008]demonstrated, analyzing monthly means of temperature for 13 RCMs, how many models share systematic temperature dependent biases. The implication of this would be that the common assumption of bias cancellation in climate change projections can have noteworthy limitations. Model deficiencies related to the feedbacks between atmosphere and land surfaces become relevant at climate time-scales. In particular soil-moisture appears to be controlling the realism of models' ability to simulate the seasonal cycle accurately in climates with a clear wet-dry seasonality characterising winter vs. summer conditions.Seneviratne et al. [2006]identified the contribution of soil moisture-temperature feedbacks in a changing climate to be a dominant factor in controlling summer temperature variability in Europe (Mediterranean and Central Europe) whileDiffenbaugh et al. [2007]linked non-linear warming of hot extremes in climate change projections for the Mediterranean region to soil moisture-climate feedbacks.Diffenbaugh and Ashfaq [2010]demonstrated strong correlation between projected changes in temperature and changes in soil moisture in climate-change simulations for North America. SimilarlyBoé and Terray [2008] have identified two main clusters in the CMIP3 ensemble [Meehl et al., 2007] based on their projected changes in evapotranspiration regime, and demonstrated that the two clusters have a fundamentally different behaviour with respect to projected changes in mean temperature and precipitation. One study using the same data as in Christensen et al. [2008]clearly identified that ENSEMBLES RCMs tend to overestimate soil moisture-temperature feedbacks in Central Europe and further to this identified asymmetric effect of soil moisture availability for hot extremes in observations in Southeastern Europe [Hirschi et al., 2011].

[4] When correcting the estimated warming in models that clearly suffers from systematic temperature dependent biases, models tend to agree on a somewhat lowered level of regional warming [Boberg and Christensen, 2012; hereinafter BC12]. In their study, BC12combined information from RCM simulations nested within the European Centre for Medium Range Weather Forecasting Reanalysis (ERA-40) [Uppala et al., 2005] with simulations of the same RCMs driven by 4 different GCMs. But they also showed that the GCMs constituting the CMIP3 [Meehl et al., 2007] data base share the same tendency for systematic temperature dependent biases identified for RCMs in Christensen et al. [2008]. It is therefore a very relevant question to ask whether the most recent generation of GCMs used within CMIP5 [Taylor et al., 2012b] also suffer from similar bias behaviour and if the ensemble of models can be used for estimating how much these deficiencies explain of the model projections of future regional temperatures. This is the subject of the present study. We note that CMIP3 differs from CMIP5 not only in model vintage, but also from the fact that the emission scenarios for the future differ. Considerable efforts have gone into the design of a more policy relevant set of scenarios, trying to keep some similarity to the previous generation of scenarios [Moss et al., 2010]. However, climate sensitivity of the CMIP5 class models appear to be comparable to those in CMIP3 [e.g., Fasullo and Trenberth, 2012; IPCC, 2007].

2. Model Configurations and Validation Data

[5] We use 26 regions recently introduced in the IPCC SREX report [Seneviratne et al., 2012] encompassing a wide range of different climate characteristics, not only the warm and dry summer conditions found in the Mediterranean region being the focus in BC12.

[6] We have accessed what was available in the CMIP5 archive before May 2012 satisfying the criteria that simulations were available for the historical period (1850–2005) and for the RCP4.5 scenario [Moss et al., 2010; Meinshausen et al., 2011]. This resulted in a total of 34 GCM simulations (and a total of 17 different models; see auxiliary material, Table S1 in Text S1). As we essentially want to follow BC12, we choose to illustrate systematic temperature dependent biases in these GCMs by plotting monthly data in a q-q diagram (seeFigure 1) showing model against observations (CRU v3.10) [Mitchell and Jones, 2005]. As climate models by construction are not meant to represent actual climate in a strict month by month sense, the only way modelled fields can be assessed are by comparing the statistical behaviour against observations. A q-q diagram is one way to capture a models performance against observations representing all modes of variability rather than merely the mean or other more simple statistics. In order not to emphasize general warm or cold biases which is not what we are interested in, model data can be centred to get a zero mean in model temperature with respect to the diagonal (seeauxiliary material Figure S1 in Text S1 and compare with Figure 2b in BC12). It is evident that in many regions models deviate upwards in the q-q diagram in the warmer months. This is equivalent to the bias behaviour identified byChristensen et al. [2008]. A few models actually show the opposite in certain regions. The 50% warmest months capture the climatic conditions that precede and include the hottest months as well as the end of the warm season capturing the period where the feedback mechanisms involving moisture fluxes between surface and atmosphere and within are strongest. Taking the deviation from a ‘perfect’ model fit to the observational records as a measure of model skill, we calculate a differential temperature bias fit defined as the slope from fitting a straight line to the 50% warmest months in the q-q diagrams represented relative to the diagonal (slope equal to 1), i.e. positive (negative) values imply an upward (downward) deviation. In the tropics the temperature span is only a few degrees, and here relatively little value may result from studying the q-q diagrams. But we have kept all regions in this analysis in order to provide a complete survey of the SREX regions [e.g.,Seneviratne et al., 2012].

Figure 1.

Ranked monthly mean temperatures for 26 subregions for 1961–2000, GCM temperatures as a function of observed temperatures. The diagonal represents TGCM = Tobs.

Figure 2.

Projected temperature change for the 50% warmest months given as a function of temperature bias slope for 26 subregions. Blue symbols represent temperature change for 2021–2050 relative to 1961–2000, whereas the red symbols denote temperature change for 2071–2100 relative 1961–2000. The temperature bias slope is defined as the slope Tbias/Tobs relative to the diagonal for the 50% warmest months extracted from Figure 1. Linear fits are added to each subregion. The inserted crosses for each panel represent an estimate of the associated st.dev. for the individual models.

3. Model Bias Characteristics and Projections

[7] Figure 2relates the temperature dependent bias for the period 1961–2000 to the degree of warming in the CMIP5 RCP4.5 experiments for two time periods (2021–2050 and 2071–2100) relative a reference period, here chosen to be 1961–2000. Each point represents the projection from individual models shown against the models' degree of temperature dependent bias. The overall impression is that in many regions a positive correlation between regional temperature increase and the q-q temperature bias fit is present. If extrapolation of the q-q diagrams into the yet unrealised warm regimes holds, models with a q-q relation that is parallel to an ideal match would provide an unbiased temperature projection, while positive (negative) values tend to overshoot (undershoot). This is exactly, what is seen inFigure 2 in many regions. Based on the linear fits to the data shown in Figure 2, Table 1 provides the estimated slope (column 2, all statistically significant at a 95% level in the regions where numbers are shown in bold), the projected temperature change value (column 3) for a potential ‘perfect’ model with zero differential bias (e.g. where the value on the horizontal axis in Figure 2is 0), and the adherent standard deviation of the projected temperature change (column 4). The temperature increase expressed for a model with no differential bias is a best estimate of the ensemble mean warming and the standard deviation of this mean warming is an estimate of the uncertainty due to inter-model uncertainty. For comparison, the non-corrected mean (column 5) and standard deviation (column 6) of the total ensemble is also provided. Overall, by taking into account that models have different temperature dependent biases the projected model mean warming (column 7) is somewhat reduced (by 10–20%) all statistically significant at a 95% level in the regions where numbers are shown in bold. In the tropics and the very highest latitudes on both hemispheres, there is no statistically significant correlation, while one region (Western Africa) exhibits a negative (but not statistically different from zero) correlation. In this single case the multi-model mean warming appears to be underestimated. Note that the observed annual temperature span is approximately 5°C in this region.

Table 1. Regional Statistics Extracted From Figure 2a
RegionSlope [°C]ΔT Slope = 0σΔT Slope = 0 inline imageσΔT1-ΔT0 / inline image
  • a

    First column lists the region. The second column gives the slope of the linear fits shown in the figures; first value for the near future and the second value for the far future. Slopes statistically different from zero (at the 95% confidence level) are shown in bold. The third and fourth columns list temperature changes deduced from the potential model with no differential temperature bias (curve parallel to the diagonal in Figure 1 and hence represented by a zero value on the horizontal axes in Figure 2) and associated standard deviations, respectively. The fifth and sixth columns give the ensembles mean temperature change and standard deviations, irrespectively the value on the horizontal axis. The seventh column shows the relative reduction (in percent) in the regional warming, where bold faced numbers represent changes statistically significant at the 95% confidence level.


[8] When comparing the relations for the two time slices (near and far future), we find that only in those regions where the correlation is positive (and statistically significant), the linear fit to the data points in Figure 2 exhibits an increasing slope with increased warming. We note that the same relative reduction (column 7) appears to result from both near term and far term projections, where the numbers are statistically significant. This indicates that the temperature biases inferred from Figure 1are systematic and preserved throughout the full temperature regime reached within the RCP scenario we have analysed. In other words, the inferred deviation from an unbiased q-q diagram is preserved as the model worlds warm up – no indication of saturation of the effect is seen.

[9] Each panel in Figure 2 is reproduced individually in Figure S2 in Text S1, where each symbol is replaced by the model number given in Table S1 in Text S1. By combining the ranking of the absolute bias slopes for each model in each region, some models stand out with a relatively high differential temperature bias on average (CanESM2, CNRM-CM5, GFDL-ESM2G, HadGEM2, MIROC5, MIROC-ESM and MRI-CGCM3) while some models stand out with a relatively small bias slope on average (EC-Earth-DMI, GISS-E2-R, NorESM1-M). This suggests that the relative high climate sensitivity of the former group is partly resulting from the systematic errors that are clearly diagnosed through their substantial differential temperature bias. In the same sense, the second group appears to be more credible.

4. Summary and Conclusions

[10] Following BC12, we have analysed how temperature dependent model biases, varying between models, influence the projected ensemble mean warming. We have found that the temperature dependent model bias influences the projection of regional change in many models. Therefore, projected warming is overestimated by these models and when taking an ensemble mean or median to provide a best estimate of the projected warming, the result is also an overestimation. We have shown that for the RCP4.5 scenario, the overestimation in the warm month temperature can be as high as 0.5–0.6°C in some regions by the end of the century. Corrected estimates are therefore 10–20% lower than the raw multi-model mean result.

[11] As also pointed out in BC12, we find the largest effect where summers (the warmest months) in general are characterized by a substantial dry period, but not necessarily every year. The regions, where this applies are typically areas where the warm period is dominated by large scale subsidence for a substantial period and therefore, we propose that models with a tendency for drying out the land surface at an early phase of the warm season is more likely to suffer from excessive warming due to various feedback mechanisms not being properly balanced. Only one region at mid-latitudes with relatively large annual amplitude in temperature, East North America turns out not to yield a statistically significant correction in model mean warming. This region is not dominated by subsidence in the warm season and therefore not prone to be affected from such a feedback loop. There is evidence that models with a wrong partitioning of latent and sensible heat fluxes controlled by soil moisture during summer time conditions are prohibitive for getting model agreement on projected change in temperature variability across multi-model experiments [Fischer at al., 2012]. This is in line with the finding by Quesada et al. [2012]who realised that models with most realistic representation of soil moisture-temperature feedbacks appear to project stronger change in temperature extremes in Europe. The global scale relevance is clearly demonstrated byMueller and Seneviratne [2012] extending the finding of Hirschi et al. [2011]to the global scale showing how soil moisture-temperature feedbacks are relevant for temperature extremes in a large fraction of the globe consistent with the findings in the present work. Parameterized processes that link with the surface moisture budget, in particular sub-grid scale processes used in convection, land-surface and cloud schemes are influencing how soil moisture-precipitation feedbacks are handled [Hohenegger et al., 2009; Lenderink et al., 2007; Fischer and Schär, 2009; Taylor et al., 2012a], and these may in turn also affect soil moisture-temperature feedbacks. It is therefore not whether models manage to represent the feedback mechanisms, but rather if there might be a threshold involved that triggers a wrong behaviour, when surpassed. This could be when simulated soil moisture approach or drops below the plant wilting point. It is beyond the purpose of this work to investigate this in detail.

[12] Overall, taking a multi-model ensemble approach to project future climate change is overcoming some of the limitation originating from using individual models known to be imperfect. However, here we have pointed out that accepting model spread as a way to portray uncertainty of the projection estimate may result in an overestimation of the projected warming and at the same time indicate little model agreement on the mean value. A non-negligible part of this is due to model deficiencies. When corrected for either from post processing as we have illustrated here or from further model improvement, lead to better inter-model agreement and therefore a more robust overall projection of regional climate change signals.


[13] We acknowledge the Program for Climate Model Diagnosis and Intercomparison (PCMDI) for making available the WCRP CMIP5 multimodel data set. We are grateful for valuable comments on this manuscript by Sonia Seneviratne, ETH Zurich. This work received financial support from the Danish Strategic Research Council through its support of Centre for Regional Change in the Earth System (CRES; www.cres-centre.dk) under contract DSF-EnMi 09-066868 and furthermore it is part of the Greenland Climate Research Centre (project 6504).