Accuracy of climate change predictions using high resolution simulations as surrogates of truth



[1] How accurate are predictions of climate change from a model which is biased against contemporary observations? If a model bias can be thought of as a state-independent linear offset, then the signal of climate change derived from a biased climate model should not be affected substantially by that model's bias. By contrast, if the processes which cause model bias are highly nonlinear, we could expect the accuracy of the climate change signal to degrade with increasing bias. Since we do not yet know the late 21st Century climate change signal, we cannot say at this stage which of these two paradigms describes best the role of model bias in studies of climate change. We therefore study this question using time-slice projections from a global climate model run at two resolutions - a resolution typical of contemporary climate models and a resolution typical of contemporary numerical weather prediction – and treat the high-resolution model as a surrogate of truth, for both 20th and 21st Century climate. We find that magnitude of the regionally varying model bias is a partial predictor of the accuracy of the regional climate change signal for both wind and precipitation. This relationship is particularly apparent for the 850 mb wind climate change signal. Our analysis lends some support to efforts to weight multi-model ensembles of climate change according to 20th Century bias, though note that the optimal weighting appears to be a nonlinear function of bias.

1. Introduction

[2] Predictions from comprehensive climate models provide the underpinning for climate mitigation policy, for adaptation strategies [Intergovernmental Panel on Climate Change, 2007a] and for quantifying geoengineering proposals [Keith, 2000]. It is crucial that such predictions, particularly changes in the climatology of regional weather variables such as precipitation and windiness, are accurate. But how can we tell if current projections are sufficiently accurate to meet the needs of policy? All climate models are biased in one way or another against observations of contemporary climate, but how much should predictions of future climate be discounted by biases against contemporary climate? At one extreme, one could discount in proportion to the bias; the bigger a model's bias the less relevant its climate change projections to the real world. At the other extreme, if model bias can be considered a state independent linear offset (and hence the same for both 20th and 21st Century climates), the simulated climate change signal would be largely independent of model bias and one would not discount even if model bias was large. In the absence of observations of late 21st Century climate change, how does one choose between these extremes, and is there a more optimal intermediate strategy?

[3] In this paper, we test this idea by running a climate model, itself based on a numerical weather prediction (NWP) model, at two different resolutions: a resolution comparable with IPCC AR4 class models, and a resolution used in contemporary NWP. The high resolution integrations are then used as surrogates of truth for both 20th and 21st Centuries. We will examine, for different regions of the world, whether the magnitude of low-resolution 20th Century model bias, computed with respect to this proxy truth, is a predictor of the accuracy of the low-resolution model's regionally-varying climate-change signal, again estimated relative to the proxy 21st Century truth.

2. Methodology

2.1. Model and Model Experiments

[4] The global atmospheric climate model used here was developed by the Japanese Meteorological Research Institute and is based on the Japan Meteorological Agency's operational spectral medium-range NWP models [Mizuta et al., 2006]. Simulations were performed at two different resolutions: TL95L40 and TL959L60. Here “TLx” refers to truncation at total wavenumber x using a triangular spectral truncation based on a Gaussian grid, and Ly refers to a vertical truncation with y levels in the vertical. For the spectral truncations above, the corresponding horizontal spatial truncation scales are 180 km and 20 km, respectively - for reference, the 180 km truncation scale is typical of state-of-the-art climate models (and in fact somewhat finer than those of typical IPCC-AR4 models [Intergovernmental Panel on Climate Change, 2007b]), whilst the 20 km truncation scale is more typical of state-of-the-art global NWP models [World Meteorological Organization, 2010]. For the TL95L40 resolution model, 3-member initial-value ensemble simulations were conducted. Insufficient computing resources prevented such an ensemble being performed for the TL959L60 model, but in any case, having only a single realisation of our surrogate truth model is no different to the situation when we compare models with reality. Unless otherwise stated, results from the TL95L40 model are based on ensemble-mean fields.

[5] A problem faced by all modelling groups as they increase resolution substantially (e.g., from 180 km to 20 km), is the fact that certain sub-grid parameters are themselves resolution dependent, and without taking this in to account, the model climate may become unrealistic in certain respects [Mizuta et al., 2006]. Hence, some changes have been made to a number of parameters in the TL959L60 model compared with the lower resolutions - see Mizuta et al. [2006] for details. As such a comparison of the two models cannot be said to denote purely the effect of different resolution.

2.2. Boundary Conditions

[6] Since it is currently impossible to run multiple century-long integrations with fully-coupled climate models at the resolution of NWP models - the climate modelling community does not have sufficient access to top-of-the-range high-performance computing - we make use here of the “timeslice” technique, whereby an atmosphere-only model is integrated over two periods of 25 years corresponding to the late 20th Century and the late 21st Century with prescribed sea surface temperatures (SSTs). We do, however, recognise that prescribed SST integrations are themselves subject to systematic biases due to one-way coupling [Douville, 2005].

[7] Model integrations were conducted for the (“control”) period 1979–2003 using observed interannually-varying HadISST SSTs and sea ice concentrations (SICs) [Rayner et al., 2003] as lower boundary conditions. For the period 2075–2099, the SST and SIC climate-change signals are estimated by the CMIP3 [Meehl et al., 2007] multi-model ensemble mean to which the detrended interannual variations in HadISST have been added [Mizuta et al., 2008]. In this way, both control and timeslice integrations are integrated with interannually varying SSTs and SICs. The IPCC SRES A1B scenario was assumed for future emissions of greenhouse gases.

3. Results

[8] Figure 1 shows the conventional “Giorgi” regions [Giorgi and Bi, 2005], used as the basis of our verifications below. Table 1 shows the 20th Century RMS bias in 850 hPa wind (U850) for all individual Giorgi regions for the low and high resolution models, for December to February (DJF) and June to August (JJA), against real data (Japanese reanalysis [Onogi et al., 2007]). Notice that in general (10 out of 16 entries in Table 1) the high resolution model has lower bias than the low resolution model against the real data - in a further 3 cases the single-member high-resolution simulation has equal bias with the smoother low-resolution ensemble-mean field. As an example, Figure 2 shows the region in the Northern Hemisphere where the difference in bias between high and low resolution models is largest in DJF. In this European region, the high resolution model performs substantially better in terms of simulation of blocking activity [Matsueda et al., 2009].

Figure 1.

Geographical domains of the world used to calculate the root mean square biases and correlations in Tables 1, 2a, and 2b.

Figure 2.

Biases in 850hPa zonal wind against observations (JRA25) based on 25-year control integrations: (a) 3-member ensemble mean of TL95L40, and (b) single integration of TL959L60 for DJF (December to February) over Europe.

Table 1. Root Mean Square Bias in 850 hPa Zonal Wind Against Observations Based on 25-Year Control Integrations for December to February and June to Augusta
RegionTL959L60TL95L40 Ensemble Mean
  • a

    Geographical definitions of regions are given in Figure 1. Center and right columns are for single integration of TL959L60 and 3-member ensemble mean of TL95L40, respectively. AF, Africa; AN, Australia and New Zealand; AS, Asia; EU, Europe; NA, North America; RU, Russia; SA, South America; TA, Tropical Asia.

RMS Bias in U850 Against JRA25 (DJF)
RMS Bias in U850 Against JRA25 (JJA)

[9] Henceforth in this study, we treat the TL959L60 simulations (the NWP model output) for both 20th and 21st Century, as a proxy for truth. Hence below, when we speak of “model”, we refer to the TL95L40 low-resolution model. The generally superior performance of the TL959L60 20th Century simulation with respect to real 20th Century data gives some justification for this methodology, though the methodology does not require this to be the case.

[10] For all regions in Figure 1 and for each of DJF and JJA, Table 2a shows the strength of the 20th Century model bias, expressed as a spatial correlation coefficient between the model fields and (proxy) truth, for time-mean U850 and precipitation, separately. For example, the Table 2a shows that Europe is a region with strongest model bias in U850 in DJF, and Russia in JJA. These two regions stand out from the others in having a correlation bias of less than 0.8.

Table 2a. Spatial Correlations of the Present 25-Year Mean of 850 hPa Zonal Wind and Precipitation Between Single Integration of TL959L60 and 3-Member Ensemble Mean of TL95L40
RegionDecember to February Correlation of the Present 25-Year MeanJune to August Correlation of the Present 25-Year Mean

[11] The key question we wish to address here, is whether, these regionally varying biases provide a predictor of the accuracy of the model's regional climate change signal. In particular, does Europe in DJF and Russia in JJA stand out as regions where the climate change signal is particularly inaccurate? Table 2b, again for all Giorgi regions, and for U850 and precipitation for DJF and JJA, shows the spatial correlation between the model's time-mean climate change signal and the (proxy) climate change truth. It is immediately seen that the model climate change signal is negatively correlated with truth for U850 over Europe in DJF, and for U850 for Russia in JJA. No other Giorgi regions show such inaccuracy in climate change signal. As an illustration, Figure 3 shows the climate change signal for U850 over Europe, for DJF, illustrating how much different the model and proxy truth climate change signal can be.

Figure 3.

Climate change signal of 850 hPa zonal wind for (a) 3-member ensemble mean of TL95L40 and (b) single integration of TL959L60 for DJF (December to February) over Europe.

Table 2b. Spatial Correlation of Climate-Change Signal Between TL959L60 and 3-Member Ensemble Mean of TL95L40
RegionDecember to February Correlation of Climate-Change SignalJune to August Correlation of Climate-Change Signal

[12] More generally, the correlation, taken over all regions, between the strength of 20th Century model bias (Table 2a), and the error in climate change signal (Table 2b) for U850, is 0.64 for DJF and 0.75 for JJA. For precipitation the correlations are 0.16 for DJF and 0.63 for JJA. If the two regions EU for DJF, and RU for JJA are removed from the U850 sample, the correlation between bias and error of climate change signal drops to −0.08 for DJF and 0.38 for JJA. This indicates that the relation between bias and climate change signal is not in fact a simple linear one.

[13] Since Europe and Russia are the two regions where the model U850 winds have largest bias, one could clearly reduce the impact of the model's erroneous climate change signal by discounting the model climate change signal based on the magnitude of the proxy bias. On the other hand, much of the correlation between bias and error in climate change signal is accounted for by these regions, and for DJF in particular, there is no further information to be had by bias weighting. For precipitation, the results are mixed – in JJA there is value in bias weighting, whilst in DJF the climate change signals appear more independent of bias.

[14] A simple conclusion from our study is that discounting should certainly be undertaken when bias exceeds a threshold of significant bias, but not necessarily undertaken for smaller biases, i.e., optimal weighting by bias might be a rather nonlinear procedure.

4. Conclusion

[15] Using high resolution simulations as a surrogate of truth, we have shown that the regionally dependent 20th Century 850 hPa zonal wind and precipitation bias of a climate model is a predictor of the accuracy of its 21st Century climate change signal. In particular, in two regions where model bias was especially large, the low-resolution model's climate change signal was negatively correlated with the true climate change signal.

[16] The results give some support to efforts to weight multi-model ensembles with bias, though our results suggest the weighting should depend nonlinearly with bias, and, for precipitation, may also depend on season. More generally the results in this paper lend support to aims to try to reduce model bias – the notion of a state independent linear bias offset is simply not tenable. A byproduct of our study was the finding that the bias of a model run at typical NWP resolution was typically smaller than that of an equivalent model run at typical climate resolution (though due to some changes of parameters, it cannot be stated unambiguously that the reduction of bias was uniquely due to resolution). Consistent with the seamless prediction methodology [Palmer and Webster, 1993; Palmer et al., 2008], we strongly recommend that a fully rigourous study of the impact of running climate models at today's NWP resolutions be made using fully-comprehensive coupled ocean-atmosphere climate models, where high-resolution ocean dynamics is also likely to be important [Shaffrey et al., 2009]. Given the demands of Earth-System complexity and the need for ensemble integrations, this would require computational facilities with sustained multi-petaflop performance, dedicated to climate prediction. Such facilities are currently unavailable to the climate modelling community [Palmer, 2005; Nature Editorial, 2008; Shukla et al., 2009]. Given the pre-eminence of the climate threat, and the need to reduce uncertainty in climate predictions, we believe this to be a matter of importance and urgency.


[17] This work was conducted under the framework of the “Projection of the change in future weather extremes using super-high-resolution atmospheric models,” led by A. Kitoh, supported by the KAKUSHIN Program of the Ministry of Education, Culture, Sports, Science, and Technology (MEXT) and the ENSEMBLES project, funded by the European Commission's 6th Framework Programme, through contract GOCE-CT-2003-505539. Our thanks to A. Scaife, J. Slingo, and J. Mitchell for helpful discussions and to A. Weisheimer, A. Kitoh, H. Kondo, S. Kusunoki, and T. Ose for their comments to earlier version of the manuscript. The integrations were performed using the Earth Simulator.