Scale space methods for climate model analysis

Authors


Corresponding author: K. Marvel, Program for Climate Model Diagnosis and Intercomparison, Lawrence Livermore National Laboratory, PO Box 808, L-103, Livermore, CA 94551, USA. (marvel1@llnl.gov)

Abstract

[1] In this paper, we introduce methods for evaluating climate model performance across spatial scales. These techniques are based on the “scale space” framework widely used in the image processing and computer vision communities. We discuss why the diffusion equation on the sphere provides a particularly attractive means of smoothing two-dimensional maps of global climate data. We establish that no structure is introduced into a map as an artifact of the smoothing procedure. This allows for the comparison of models and observations at multiple scales. As a test case for these methods, we compare the ability of high- and low-resolution versions of the Community Climate System Model (CCSM) to simulate the seasonal climatologies of surface air temperature (TAS), sea level pressure (PSL), and total precipitation rate (PR). For TAS, we find that the high-resolution model is better able to capture the boreal summer (JJA) climatological pattern at fine scales, although there is no such improvement in winter (DJF). We find the performances of the high- and low-resolution models to be similarly capable of capturing the summertime sea level pressure climatology at all scales. However, the high-resolution model PSL climatology is degraded for DJF, especially at larger scales. For both JJA and DJF precipitation climatologies, we find larger precipitation errors in the high-resolution model at the finest scales; however, performance at larger scales is improved.

1 Introduction

[2] In recent years there has been a concerted drive toward higher resolution in climate modeling. Increasing spatial resolution is thought to be desirable for two main reasons. First, accurate simulation of local and regional phenomena is crucial in assessing the consequences of future climate change. Variables such as precipitation are strongly dependent on small-scale processes, and models that resolve such processes and incorporate more realistic orography might provide better guidance in planning for weather extremes. Second, the large-scale dynamics of the climate system are in part determined by nonlinear processes whose characteristic scales are not resolved by the current generation of global climate models. There is evidence that explicitly resolving, rather than parameterizing, these processes may improve the fidelity of atmospheric and ocean models [Hack et al., 2006].

[3] However, finer spatial resolution does not universally lead to better performance. Simulating precipitation has proven to be especially difficult, with little evidence that higher resolution improves performance. Masson and Knutti [2011] find that higher model resolution in the Coupled Model Intercomparison Project Phase 3 (CMIP3) ensemble does not lead to better simulation of large-scale precipitation climatologies. Regionally, Rauscher et al. [2010] find that improvement in precipitation with resolution is dependent on season and geography, while Caldwell [2010] finds that improved model resolution does not translate into improved simulation of California wintertime precipitation.

[4] Simply increasing horizontal model resolution, even if it leads to better representation of small-scale processes, is of limited use if it degrades the ability of models to capture large-scale features or global mean states. It is therefore important to examine the performance of models at multiple spatial scales. The goal of this paper is to develop a consistent framework for smoothing spatial maps of climate data and to apply these methods to model evaluation. Here “smoothing” refers to a process by which variability on fine spatial scales is systematically removed to generate filtered, coarser-resolution versions of the data. This process can help in understanding the persistence of certain features in data across spatial scales, and it can allow for the evaluation of model performance as a function of spatial scale. However, in order to compare models with each other or with observations at different scales, we need confidence that the process by which fine scales are removed does not itself introduce spurious structure into the map. The “scale space” methods presented here draw upon the physics of diffusion and inherit several useful properties from the mathematical framework of the diffusion (or heat) equation. In particular, barring small errors resulting from discretization, no additional structure is introduced as an artifact of this smoothing procedure.

[5] As a demonstration case for these methods, we evaluate the ability of low- and high-resolution versions of the Community Climate System Model (CCSM) to capture the characteristic spatial variability associated with the seasonal climatological patterns of three variables. Surface-air temperature (TAS), sea level pressure (PSL), and total precipitation rate (PR) are chosen because, as two-dimensional model output fields, they provide a useful test bed for spatial analysis methods defined on the sphere.

[6] This paper is structured as follows: in section 2, we introduce a method for smoothing maps of global climate data and apply it to a high- and low-resolution version of CCSM, described in section 3. We use this smoothing procedure to create Taylor diagrams showing the changes in model spatial variability and pattern correlation as a function of the smoothing parameter; these are presented in section 4. In section 5, we show how this smoothing procedure can be adapted to a simple filtering method, which allows us to examine model performance at specified spatial scales. We introduce a succinct method of summarizing model performance at differing scales, the equal-variance Taylor diagram. In section 6, we summarize the performance of the two models according to these metrics and place our results in the context of existing high-resolution modeling literature.

2 Methods

[7] In this section, we show that the mathematics of the diffusion or heat equation provide a suitable framework for multiscale analysis. Intuitively, the process of diffusion removes fine-scale structure as a function of time, with smaller structures disappearing before larger structures. The mathematical framework underlying the diffusion process can be understood using two basic principles. First, the diffusive flux should have a magnitude proportional to the spatial gradient of the concentration and should flow from high-concentration areas to low-concentration areas. This is Fick's first law of diffusion:

display math(1)

where L(x,y;t) is the concentration at time t, j the diffusive flux, and D a constant diffusion coefficient.

[8] The second principle is a conservation law: The total amount of the quantity L(x,y;t) in a region may change only by the amount of diffusive flux passing into or out of that region. This is simply the continuity equation:

display math(2)

[9] Combining ((1)) and ((2)) gives the familiar diffusion or heat equation:

display math(3)

subject to the initial (temporal) condition

display math(4)

where f (x,y) is the initial concentration.

[10] Clearly, the result of increasing time in the diffusion framework is the removal of structure at progressively larger scales. The rate at which progressively larger scales are smoothed out is controlled by the diffusion coefficient D, which must have units of length squared per time. This suggests that we can reinterpret the product Dt as a spatial smoothing parameter; namely, it is the characteristic area of structures that are removed after diffusion acts for a time t. The removal of fine-scale structure by diffusion is the defining feature of a framework known as “scale space,” widely used in image processing [see, e.g., Lindeberg, 1993; Koenderink, 1984, and references therein].

2.1 Scale Space Implementation in Cartesian Space

[11] Suppose we begin with a map f=f (x,y), where f is the value of some climate variable such as temperature or precipitation at spatial location (x,y). We wish to construct a smoothed representation of the map f, which we will denote L(x,y;t), as a function of a single parameter t. In order to construct a scale space representation of the initial map f (x,y), we require L(x,y;t) to solve the diffusion equation ((3)) subject to the initial condition ((4)). To solve an equation of the form ((3)), we find the Green's functions [Arfken et al., 2012], defined such that

display math(5)

where inline image is the differential operator and δthe Dirac delta function. The Green's function for the operator inline image in two-dimensional flat space is a simple Gaussian:

display math(6)

and we can write the solution as

display math(7)

This means the diffused map L(x,y;t) at time t is obtained by simply convolving the original map f (x,y) with a two-dimensional Gaussian kernel given by equation ((6)).

[12] Thus, allowing an initial concentration to diffuse for time t eliminates structure on length scales of order inline image. The parameter σhas units of length and will be referred to hereafter as the smoothing parameter.

2.2 Advantages of the Scale Space Approach

[13] Evolving data forward in time using the diffusion equation allows us to construct a continuous hierarchy of representations of the data at different spatial scales parameterized by the length scale σ. This “scale space” representation is widely used in image processing because the features of the diffusion equation give rise to several useful properties unique to the Gaussian kernel [Babaud et al., 1986]. These include the following:

  • [14] Continuous parametrization. Unlike Fourier analysis, where we are restricted to discrete integer frequencies, the smoothing parameter σmay take any positive value.

  • [15] Automatic window selection. It is also possible to remove small spatial scales by imposing a low-pass filter in spatial Fourier space. However, in order to minimize side effects, the filter must utilize an appropriate windowing function, which may vary with scale. Because fine-scale structure is removed here via diffusion, the filter shape is a constant across scales, with the Gaussian width the only free parameter.

  • [16] Fast implementation. Because the smoothed representation at parameter σ given by equation ((7)) is a convolution of the original data with a Gaussian function, it can be rapidly implemented in Fourier space as a multiplication and then transformed back. This speed is particularly useful when handling high-resolution data sets.

  • [17] Causality. The diffusion equation is causal, meaning that all structures present in a large-scale representation must have their origins in the finer-resolution representation. The continuity equation ((2)) ensures that no information is introduced in the smoothing procedure. Thus, a smoothing process based on diffusion introduces no new structure into the map. In particular, local extrema are not enhanced in the smoothing [Koenderink, 1984]. This is simple to show. With inline image, the diffusion equation ((3)) becomes

    display math(8)

    At a local maximum (minimum), ∇2L(,σ) is negative (positive) definite. Because the smoothing parameter σ>0, this guarantees that the diffusion process increases the value of a local minimum and decreases the value of a local maximum. Thus, local extrema are not enhanced when structure is removed via diffusion.

  • [18] Cascade property. The smoothed representation at scale σis obtained as the convolution of the data with a Gaussian of variance σ2. Because the convolution of two Gaussians is a Gaussian, concatenating two smoothing procedures with parameters σ1 and σ2is equivalent to smoothing with a single broad Gaussian with width inline image.

[19] In conclusion, the realization of smoothing through a diffusion process is simple to implement, introduces no spurious structure into a map, and has other desirable properties listed above. By contrast, consider a smoothing process that removes fine-scale structure by truncating the Fourier representation of data at some cutoff frequency. This is equivalent to multiplying the Fourier representation of the data by a Heaviside step function. Upon transforming back to the space domain, the data will have been convolved with the Fourier transform of the step function, which introduces spurious “ringing” artifacts into the data. The use of the diffusion equation guarantees that no such artifacts are present in the scale space representation of smoothed data.

2.3 Previous Applications of Scale Space to Climate Data

[20] Scale space analysis has been used the image processing and computer vision communities for over 20 years, but it first appears in the statistics literature with the SiZer (significant zero crossings of derivatives) method for detecting statistically significant scale-dependent features in a one-dimensional field [Chaudhuri and Marron, 2000]. This method was extended by Erästö and Holmström [2005], who incorporated Bayesian methods for identifying multiscale features. These methods have found applications in the analysis of climate data [Divine et al. 2007; Godtliebsen et al., 2003; Korhola et al., 2000; Rohling and Pälike, 2005] such as the reconstructed past temperature record. In these applications, “scale” is generally temporal, and variations on scales ranging from decadal to millennial may be identified by the method. A useful review of such techniques may be found in Holmström [2010]. Other authors have extended the scale space approach to the spatial domain [see, e.g., Holmström et al., 2011; Pedersen et al., 2008], performing smoothing operations on 2-D spatial fields to identify features that are robust across different climate models at multiple scales. In this paper, we build on these foundations to introduce a simple and computationally efficient smoothing kernel applicable to the surface of a sphere.

2.4 Gaussian Equivalents on the Sphere

[21] Two-dimensional climate fields like surface temperature, precipitation, or sea level pressure are, of course, defined on the sphere, and the curvature of the Earth must be taken into consideration. In this case, the Gaussian kernel given in equation ((6)) is inappropriate. If it is simply converted to spherical coordinates (θ,φ) using the familiar Cartesian conversion formulae, the resulting function is not separable and may not be implemented as a convolution kernel—a serious efficiency consideration when working with large data sets. Moreover, the convolution of two such “Gaussians” is not itself a “Gaussian,” and the cascade property no longer holds. In their spatial-scale analysis of the CMIP3 model data, Masson and Knutti [2011] implement spatial smoothing on the sphere as a weighted average

display math(9)

with dij(x,y) the great-circle distance between points (x,y) and (i,j), a method also used by Räisänen and Ylhäisi [2011]. This method is computationally expensive and therefore somewhat unsuitable for high-resolution data sets. However, the requirement that the smoothing kernel at a point (i,j) should be symmetric along the geodesic circle centered around (i,j) can be satisfied by an isotropic diffusion process on the unit sphere. We will now show that the scale space framework above may be extended to yield a simple convolution kernel for smoothing on the sphere. This method inherits the useful properties discussed in the previous section and is simple and efficient to implement.

[22] The flat-space convolution kernel equation ((6)) is obtained as the Green's function for the diffusion operator in flat space. This suggests an extension to the sphere: the convolution kernel for smoothing is the Green's function for diffusion on the unit sphere. Given an initial data set f (θ,φ) on the unit sphere, we can calculate the smoothed representation L(θ,φ,t) by solving the diffusion equation in spherical coordinates (θ,φ), where θ is latitude and φlongitude:

display math(10)

where

display math

is theLaplacian in spherical coordinates. The Green's functions of the spherical diffusion operator are known [see Bulow, 2004; Chung, 2006], and they are given by

display math(11)

where P(sinθ) are the Legendre polynomials. As before we set inline image; here σ is an angular scale measure indicating the latitudinal extent of structures removed in the smoothing process. When σ=t=0, equation ((11)) reduces to the spherical delta function

display math(12)

meaning that applying the smoothing operation with zero smoothing parameter to the original map will leave it unchanged. Like the flat-space Gaussian kernel ((6)), this function approaches a constant for large σ.

[23] An equivalent of the convolution theorem holds in spherical harmonic space. Any square-integrable function f (θ,φ) on the unit sphere can be expanded in terms of the complete orthonormal basis of spherical harmonics Ym(θ,φ). The expansion coefficients fmare given by inline image. Driscoll and Healy [1994] prove that for any two square-integrable functions f (θ,φ) and h(θ,φ) on the sphere, the coefficients of the convolution in spherical harmonic space are given by

display math(13)

This means that the spherical harmonic representation of the smoothed data at scale σ has a simple form:

display math(14)

Equation ((14)) means the convolution is extremely simple to implement in the spherical harmonic domain. Once the spherical harmonic representation of the data has been obtained using a utility such as NCAR's SPHEREPACK [Adams and Swarztrauber, 1999], the representation of the data at scale σ is simple to calculate. This allows us to construct a scale space hierarchy of images by convolving the data with spherical “Gaussians,” defined by equation ((14)), of progressively increasing width. The initial scale is chosen to be σ=θ0, where θ0is the latitudinal resolution of the unfiltered data on the native grid. Figure 1 shows the result of this smoothing process applied to the observed TAS climatology for σ=2°,10°, and 20° latitude. Note that structures on scales less than σare almost entirely removed as a result of the smoothing process.

Figure 1.

Smoothed representations of the observed DJF TAS climatology. From bottom to top: the unsmoothed climatology, and the representations at σ=2°, 10°, and 20° latitude. As the smoothing parameter σ increases, features on scales less than σ are removed.

[24] To quantify areas of agreement between models and observations, we can use scale space smoothing to generate representations of modeled and observed data at multiple spatial scales. Because the smoothing procedure is computationally efficient, introduces no spurious structure, and can be applied to both model and observational data, it provides a useful framework for model evaluation.

3 Model Descriptions

[25] As a demonstration case for these methods, we shall now compare results from low-resolution and ultrahigh-resolution simulations of the Community Climate System Model version 4 (CCSM4). The components of the prototype version of CCSM4 used are the Community Atmosphere Model version 3.5 (CAM) [Collins et al., 2006] featuring a finite volume dynamical core [Lin, 2004], the Community Land Model version 3.0 (CLM3) [Dickinson et al., 2006], the Parallel Ocean Program Model version 2.0 (POP) [Dukowicz and Smith, 1994], and the Los Alamos Sea Ice Model version 4.0 (CICE) [Hunke and Dukowicz, 1997], with coupler version 7.0. For the 100 year low-resolution simulation, CAM and CLM are on a longitude-latitude grid with size 288×181 (∼1.25°×1° horizontal resolution) and 26 vertical levels in the atmosphere. POP and CICE are on a dipole grid (North Pole displaced in Greenland) with size 320×384 (with a nominal resolution of ∼1° at the equator) and 40 vertical levels in the ocean.

[26] In the 20 year ultrahigh-resolution simulation performed by McClean et al. [2011], CAM and CLM are on a longitude-latitude grid with size 1152×768 (∼0.25° horizontal resolution) and 26 vertical levels in the atmosphere. POP and CICE are on a tripole grid (two northern poles located on land in Siberia and Alaska) with size 3600×2400 (∼0.1° horizontal resolution). The vertical grid in the ocean includes 42 vertical levels and partial bottom cell topography [Murray, 1996]. CAM is initialized from an earlier 0.5° simulation. POP is initialized with temperature and salinity from World Ocean Circulation Experiment (WOCE) climatology.

[27] The quarter degree grid resolution of the atmospheric domain in the high-resolution simulation permits formation of tropical storms [McClean et al. 2011] and other weather features not seen in lower resolution models. The ∼0.1° horizontal resolution of the ocean domain explicitly resolves mesoscale features such as eddies, while eddy-mixing processes are parameterized in the low-resolution simulation [Gent and McWilliams, 1990] as subgrid scale processes. Even more beneficial than resolving the mesoscales in each of the individual model components is the opportunity to resolve the interaction between them at these scales and consequently to improve the effects of the feedback mechanisms [Large and Danabasoglu, 2006].

[28] To evaluate the performance of these models, we use two observational data sets: the ERA Interim reanalysis data set, spanning 1989–2009 (ERA-INT) [Dee et al. 2011], and the Global Precipitation Climatology Project (GPCP) [Adler et al. 2003] precipitation data set for 1979–2009. ERA-INT reanalysis is available on a uniform latitude-longitude grid with a resolution of 1.5°, while the GPCP grid has a resolution of 2.5°. Both of these observational data sets exist on a coarser grid than the low-resolution model. Figure 2 shows a portion of all four grids considered in this paper.

Figure 2.

High-resolution simulated TAS DJF climatology displayed using the four grids considered in this paper. Clockwise from top left: GPCP, ERA-INT, low-resolution CCSM, and high-resolution CCSM.

4 Incorporating Scale Information into Taylor Diagrams

[29] How does model performance change as a function of smoothing scale? We expect the root mean square (RMS) error to decrease as the smoothing parameter is increased, provided the observations are smoothed to the same scale [Räisänen and Ylhäisi, 2011]. This is because there are effectively fewer spatial samples in a smoothed map, as the smoothing procedure itself removes information while adding no new structure. In order to increase computational efficiency, a smoothed map may therefore be regridded to a coarser grid without loss of information.

[30] Although the decreases in RMS error and spatial variability are monotonic with increasing σ, the rate at which they change is not the same for all variables. Climate variables such as precipitation, in which most of the spatial variability occurs on fine scales, will exhibit a sharp drop in spatial variance as the smoothing parameter is increased. The reference height temperature (TAS), on the other hand, is dominated by large-scale variability patterns, in particular, the seasonal cycle and the pole-equator temperature gradient. This means the spatial variance should not decrease as rapidly as the smoothing parameter is increased.

[31] Because RMS error decreases as a function of smoothing scale, one possible metric for model performance might be the value of the smoothing parameter at which the error falls below some predetermined cutoff. Additionally, comparing the changes in spatial variability with smoothing parameter in models and observations can provide a useful diagnostic. Thus, to investigate model performance as a function of smoothing parameter, we can display information on a Taylor diagram [Taylor, 2001], which succinctly summarizes different aspects of model performance. In the following steps, we show how such a scale diagram may be constructed. For clarity, the symbols used are defined in Table 1.

  1. [32] First, we obtain smoothed representations of the observational data at a set of smoothing scales {σ0,...σN}. We set σ0= θ0, where θ0is the native latitudinal resolution of the observational grid. This yields a hierarchy of representations of the observations at different spatial scales.

  2. [33] Considering the observed field, which is the “reference,” we calculate the spatial standard deviation, O(σ), at each value of σ. We normalize O(σ) by the unsmoothed spatial standard deviation O(0). Because spatial variance decreases with smoothing parameter, O(σ)/O(0) is strictly less than or equal to 1.

  3. [34] On a conventional Taylor diagram, the reference standard deviation (in this case, the normalized standard deviation of the unsmoothed observational map O(0)) is plotted as a star. A black (unit) semicircle intersecting the star is drawn to represent constant standard deviation (see Figure 3).

  4. [35] Colored semicircles are then drawn at radial distance O(σ)/O(0), one for each smoothing scale considered. The spacing between the semicircles is a measure of the rate at which spatial variance decreases with smoothing scale, while the color indicates the value of the smoothing parameter σ.

  5. [36] For each model, we obtain smoothed representations of the relevant seasonal climatology, using the same set of smoothing parameters as applied to the observations. In order to calculate the spatial correlations with observations and standard deviations, we re-grid the models to the observational grid using the standard area-weighted CDAT regridder [Taylor, 1996], which preserves the areal average.

  6. [37] For each model, we calculate the smoothed representation of the model data at scale σand its spatial standard deviation M(σ). We normalize these standard deviations by O(0) and calculate the pattern correlation.

  7. [38] The standard deviation and correlation are then plotted in the standard way on the Taylor diagram. The distance of each point from the origin is proportional to the model spatial standard deviation, and the cosine of the azimuthal angle is the correlation with the smoothed observational pattern. Here circles represent the high-resolution model, while squares indicate the low-resolution model. The color indicates the value of the smoothing parameter σ. For added clarity, the symbol is filled if the model spatial standard deviation is higher than the corresponding reference standard deviation and unfilled if the model spatial standard deviation is lower than the reference.

Table 1. Symbols Used in the Construction of the Scale-Aware Taylor Diagram
SymbolDefinition
σiith smoothing parameter
σ0initial value of the smoothing parameter
θ0latitudinal resolution of the observational grid
O(σi)spatial standard deviation of the observed field,
 smoothed with parameter σi
O(0)spatial standard deviation of the unsmoothed
 observational field
M(σi)spatial standard deviation of the model field,
 smoothed with parameter σi
Figure 3.

Taylor diagrams for the (top row) DJF and (bottom row) JJA climatologies of (a, b) TAS, (c, d) PSL, and (e, f) PR. The semicircles represent the spatial standard deviation of the smoothed observations, normalized to the unsmoothed spatial standard deviation and colored according to the smoothing parameter σ. The corresponding smoothed representations of the low-resolution (square) and high-resolution (circle) model climatologies are calculated and plotted in the standard way on the Taylor diagram. The cosine of the azimuthal angle is the pattern correlation of the smoothed model with the smoothed observations, and the model spatial standard deviation is proportional to the distance from the origin. The (centered) RMS error with respect to the smoothed observations is calculated as the distance between the point and the intersection of the corresponding colored semicircle with the abscissa. The color scale represents the value of the smoothing parameter.

[39] Thus, the Taylor scale diagram shows how variance in the observational data set decreases as scales are removed and allows for an assessment of model performance as a function of smoothing scale.

[40] Due to computational constraints, only 20 years of high-resolution model output are currently available, while we have access to a century of low-resolution data. In the following diagrams, the seasonal climatologies are calculated using the full available period in each model. The results are relatively insensitive to the time period chosen, as we discuss in section A.

4.1 Model Bias

[41] The centered root mean square (cRMS) difference between models and observations at smoothing scale σ is obtained as the distance between the point and the point at which the corresponding reference standard deviation semicircle intersects the horizontal axis.

[42] As σincreases, the map approaches its spatial mean and the spatial standard deviation approaches zero. Thus, we expect convergence in the lower left-hand corner of the diagram for large values of the smoothing parameter. As such, Taylor diagrams contain no information about the overall model bias (i.e., the difference between modeled and observed spatial means), which is independent of the smoothing scale. Tables 2and 3indicate that the low-resolution model has lower bias for TAS and PSL in both seasons. The PR bias, while large compared to TAS and PSL biases, is similar for both models.

Table 2. DJF Climatological Biases
VariableLow ResolutionHigh Resolution
TAS (K)0.1−0.5
PSL (hPa)−.03−.4
PR (mm/d)0.30.3
Table 3. JJA Climatological Biases
VariableLow ResolutionHigh Resolution
TAS (K)0.10.4
PSL (hPa)−.2−.5
PR (mm/d)0.30.3

4.2 Surface Air Temperature

[43] Figures 3a and 3b show that both the high- and low-resolution models are “better” at capturing observed TAS climatologies than any other variable considered. This is because global temperature patterns are dominated by well-understood sources of large-scale spatial variability, namely, the seasonal cycle and the equator-pole temperature gradient. For both DJF and JJA climatologies, the pattern correlation exceeds 0.99, even for the smallest values of σ. For the DJF climatology, the spatial variability exceeds the observed spatial variability at all scales in both models. For the JJA map, however, only the low-resolution model overestimates spatial variance at all values of σ. The high-resolution model overestimates spatial variance only at intermediate (6°<σ<28°) values of the smoothing parameter.

4.3 Sea Level Pressure

[44] Figures 3c and 3d indicate that the centered RMS error in both models is larger for PSL than TAS. This is not surprising, because the processes involved in simulating sea level pressure are less strongly influenced by direct forcing. The intermodel differences are most pronounced for the boreal wintertime PSL climatology. The low-resolution model overestimates the amplitude of spatial variability for σ<26° and underestimates it for larger σ. The high-resolution model yields similar results, but with the transition between overestimation and underestimation occurring at a larger smoothing scale σ≈34°. The pattern correlation for the low-resolution model is strikingly better for all values of σ, contributing to a smaller overall cRMS error. In contrast, there is very little difference between models for the JJA climatology. Both models overestimate the spatial variability at all smoothing scales for the JJA PSL climatology, and the similarity between high- and low-resolution model performance is evident.

4.4 Precipitation

[45] Large spatial variations on the order of a single grid cell may lead to large errors in the precipitation climatology maps. The black symbols in Figures 3e and 3f indicate that the centered RMS error of the unsmoothed high-resolution map exceeds that of the low-resolution map for DJF PR. The errors are comparable for JJA PR. In both cases, the pattern correlation for the high-resolution model exceeds that of the low-resolution model, and the larger error is due to the fact that the amplitude of spatial variability is larger in the high-resolution model climatology. The spatial standard deviation of the high-resolution model drops by over 11% upon the first smoothing, while the spatial standard deviation of its low-resolution counterpart changes by less than 8%. For JJA, the high-resolution model has more spatial variability at finer scales, but this overestimate is compensated for by superior pattern correlation with the observations, leading to a comparable centered RMS error.

5 Isolating Scale Patterns Using Taylor Diagrams

[46] The smoothing process described above progressively removes scales, generating a new, more coarsely resolved picture of the input data. It thus acts as a low-pass filter, removing variability at small spatial scales. Small values of the smoothing parameter yield maps that contain information on more spatial scales, and when σ is set to zero, we recover the original map. This means that errors at large scales (or even model global mean bias) will affect the unsmoothed map and persist even as small scales are removed by the blurring process. For many applications, however, it is important to isolate certain scales and to examine model performance independent of bias or large-scale structures. The difference of two low-pass filters with parameters σ1 and σ2yields a bandpass filter [Witkin, 1984], preserving only angular scales σ1<σ<σ2. Because convolution is a linear operation, these filters may be implemented by either applying a “difference of Gaussians” (DoG) filter of the form

display math(15)

or by subtracting the scale space representations of the data at scales σ1 and σ2.

[47] The filter boundaries in equation ((15)) may be chosen to isolate any set of scales. How, then, do we determine a useful set of scales at which to examine the climatological patterns? To choose these boundaries, we rely upon the fact that Taylor diagrams display the standard deviation of the models with respect to the observational standard deviation. If we choose a set of filter boundaries that divide the observations into n components, each of which contains equal spatial variance, then the n filtered model components may be plotted with respect to the same point on a Taylor diagram. This equal-variance partition is chosen in order to succinctly display multiple results on the same plot, and hence the choice of scale boundaries is necessarily somewhat arbitrary. The issue of scale boundary choice has received some attention in the literature (see Holmström et al. [2011], for instance), and smoothing levels may be chosen according to alternative criteria such as density of detectable features. However, a simple equal-variance partition is useful for model-observation comparisons, as we demonstrate below.

  1. [48] Using scale space bandpass filters, we resolve the observed field into n components, with each component capturing an equal fraction of the total spatial variance. Component 1 corresponds to the finest scales, component n to the largest scales. In this example, we choose n=4. The component boundaries are contiguous so that together with the global spatial average, the components sum to reconstruct the original field. It is important to note that the variances of the individual components do not sum to the total variance; in general, we expect nonzero covariance between different components (i.e., they are not orthogonal). Tables 4and 5show the variance in each component, as well as the upper bounds (in degrees latitude) for each component. The upper bounds of component 1 are far smaller for the PR climatological maps than they are for the TAS maps, indicating that small-scale features contribute more to the overall PR spatial variance than to the TAS spatial variance. The spatial variance on scales greater than the upper boundary of component 4 is negligible. This means that smoothing the data with parameters greater than the component 4 upper boundary yields the global mean. For instance, Table 4indicates that there is effectively no further spatial variance in the DJF PR climatology map on scales larger than about 100°.

  2. [49] We then partition the seasonal climatologies generated by each model into four components, where the scale boundaries of each component are the same as for the observations.

  3. [50] The standard deviation and the pattern correlation in each component are then calculated relative to the corresponding observed component.

  4. [51] Because the observed scale boundaries are constructed such that each component captures an equal fraction of the total spatial variance, all model results may be plotted with respect to the same standard deviation contour—the black semicircle in Figure 4. Color is used to denote the spatial scale, with cooler colors denoting fine scales, and warmer colors indicating large scales. The white symbols indicate the overall performance of the spatially unfiltered model output.

Table 4. DJF Component Values
 TASPSLPR
Spatial standard deviation in each component5.6 K3.4 hPa0.7 mm/d
Component 1 upper bound (deg)13205
Component 2 upper bound (deg)242212
Component 3 upper bound (deg)384223
Component 4 upper bound (deg)114124101
Table 5. JJA Component Values
 TASPSLPR
Spatial standard deviation in each component6.1K3.4 hPa0.75 mm/d
Component 1 upper bound (deg)1284
Component 2 upper bound (deg)221810
Component 3 upper bound (deg)383322
Component 4 upper bound (deg)137113123
Figure 4.

Equal-variance diagrams constructed by spatially filtering the seasonal climatology into four components, each capturing an equal fraction of the total spatial variance. Diagrams are shown for (top row) DJF and (bottom row) JJA climatologies of (a, b) TAS, (c, d) PSL, and (e, f) PR. This allows each component's pattern correlation, spatial standard deviation, and RMS error to be plotted on the same scale. The low-resolution model results are plotted as squares, while the high-resolution results are circles. Component 1, which contains the finest scales, is colored dark blue, component 2 is cyan, component 3 is yellow, and component 4, which contains the largest-scale pattern, is red. For comparison, the unfiltered model outputs are plotted as white symbols with the appropriate scaling.

5.1 Surface Air Temperature

[52] The appearance of the map in each component reflects the physical processes that dominate at the scales in question. For example, consider Figures 5b–5e, which show the filtered patterns in each of four components for the DJF TAS climatology. The fine-scale pattern in component 1, corresponding to scales between 1.5° and 12.4°, is largely dominated by orography, but also shows the strong contrast between continental and marine temperatures, especially in the Northern Hemisphere. In component 2 (fine-to-intermediate scales) we see cold features in the high-latitude northern and southern hemisphere indicating a large gradient along the polar front, as well as land/ocean contrast. Component 3 is dominated by the spatial pattern associated with the equator-to-pole temperature gradient. Finally, component 4 (largest scales) reveals the simple seasonal large-scale pattern: the winter hemisphere is colder than the summer hemisphere, although the equator-to-pole temperature gradient is still present.

Figure 5.

Equal-variance decomposition for (left column) DJF and (right column) JJA surface-air temperature climatologies calculated from ERA-INT reanalysis. (a, f) seasonal mean, (b, g) component 1 (finest scales), (c, h) component 2, (d, i) component 3, and (e, j) component 4 (largest scales). The components are chosen such that the spatial variance in each component is equal (to within ±1%).The scale boundaries and spatial standard deviations of each component are listed in Tables 4 and 5.

[53] Figure 4a shows that both models underestimate the amplitude of the DJF TAS spatial variability at the finest scales, but overestimate the variability in the larger-scale components 2–4. Model simulation of the largest-scale pattern (component 4) is inferior to simulation of the slightly smaller scales represented by component 4. This implies that both models overestimate the strength of the seasonal cycle in boreal wintertime, leading to slightly exaggerated large-scale variability. For the JJA TAS climatology, as shown in Figure 4b, both models underestimate fine-scale spatial variability, but both pattern correlation and variability amplitude are superior for the high-resolution model. The large-scale variability is better simulated in boreal summer than in winter, and the smallest errors occur at the largest scales.

5.2 Sea Level Pressure

[54] As with the temperature component maps, the sea level pressure components shown in Figure 6 reveal the signatures of the different physical processes that dominate at various scales. At the fine scales shown in component 1, orographic effects are apparent, as are smaller features within discrete areas of high or low pressure. In component 2, the broader outlines of these features become apparent in both DJF and JJA climatologies, as does the zonal pattern in the Southern Hemisphere. For the DJF PSL climatology, these discrete pressure anomalies disappear at the larger scales shown in component 3, although the Northern Hemisphere high-pressure anomalies remain in the JJA climatology. Finally, component 4 depicts the largest-scale patterns: the meridional pressure gradient decreasing from north to south and an east-west gradient in the JJA climatology.

Figure 6.

As in Figure 5, but for sea level pressure.

[55] The scale-aware Taylor diagrams presented in the preceding section indicate that the intermodel differences at simulating PSL climatologies are pronounced in boreal winter, but very small in the summer. Figure 4c shows that in the high-resolution model, the amplitude of DJF spatial variability exceeds the observed and low-resolution variability for all but the largest spatial scales. Surprisingly, the centered RMS error of the high-resolution model is larger at intermediate scales (components 2 and 3) than at the finest scales (component 1). Additionally, both models exhibit similar performance at the finest scales, indicating that the degradation in performance in the high-resolution model is not attributable to increased error at small scales.

[56] For the JJA climatology, the Taylor diagram in Figure 4d reinforces the similarity between the models observed in the scale-aware Taylor diagram. The variability at the finest scales (components 1 and 2) is underestimated (albeit very slightly for component 2), while variability at the largest scales is overestimated.

5.3 Precipitation

[57] The PR equal-variance decompositions for DJF and JJA are shown in Figure 7. For both seasonal climatologies, the fine-scale PR pattern in component 1 is dominated by orography and by the thin band of increased precipitation at the Intertropical Convergence Zone (ITCZ). In components 2 and 3, the tropical wet zones and subtropical dry zones become apparent, as do the midlatitude storm tracks. Finally, in component 4, the largest-scale patterns indicate that the poles receive less precipitation and that the tropical western Pacific receives more precipitation than the east.

Figure 7.

As in Figure 5, but for precipitation based on the GPCP data set.

[58] Figure 4e shows the resulting Taylor diagram for DJF PR. At the finest scales, both models exhibit nearly identical DJF pattern correlation, but the low-resolution version has smaller centered RMS error. This is because the high-resolution model exhibits excess spatial variability on the scale of a single grid cell, as seen in the previous section. We note, however, that both models overestimate the amplitude of PR spatial variability in all components. At the larger scales, model performance is similar. Figure 4f shows the equal-variance Taylor diagram for JJA PR, in which the results are similar. We again see nearly identical pattern correlation for the high- and low-resolution models in Component 1. For both seasons, the spatial variability of the high-resolution model exceeds the spatial variability of the low-resolution model in all components.

6 Summary of Model Results

[59] Advances in high performance computing have made it possible to realize ultrahigh-resolution in fully coupled global climate simulations, advancing toward a new generation of weather-scale climate models. The accurate validation of these fine-grid model results is difficult since most of the available observational data sets with global coverage are on coarser grids. This demands the development of methods to examine the fidelity of the high-resolution model output at the spatial scales resolved by the observations.

[60] In this study, we have introduced a consistent framework of spatial smoothing that allows us to evaluate and compare the performance of climate model simulations at various spatial scales. We applied this methodology to an ultrahigh-resolution prototype CCSM4 simulation. Our results complement and further quantify the bias analysis of the high-resolution model simulation reported previously in McClean et al. [2011]. In order to assess the effect of the increased model resolution (0.25°), these results were compared to a lower resolution (1°) version of the same model. Similar resolution-dependent studies with earlier versions of CCSM were carried out by Hack et al. [2006] with T 42×1 and T 85×1 CCSM3 and by Gent et al. [2010] with 2° and 0.5° CCSM3.5 where the authors compared the biases in the mean climate of the simulations, focusing on certain areas of interest. Here we provide a different perspective for the intermodel comparison by assessing model performance at simulating climatologies at different spatial scales.

[61] We examined the DJF and JJA seasonal climatology of three atmospheric fields: surface air temperature, sea level pressure, and precipitation. The performance of both models was evaluated first as a function of smoothing parameter, and subsequently at four scale ranges, in terms of RMS error, spatial standard deviation, and pattern correlation with the observations. The analysis shows that the “best” simulated (closest to the observed) variable considered is TAS; this is the case in both models. The high-resolution model outperforms its low-resolution counterpart in JJA, particularly at the finest scales. No substantial difference is found in the model performance in the DJF pattern. The latter result agrees with the finding of Masson and Knutti [2011] that the finer-resolution models in the CMIP3 ensemble do not seem to improve the agreement with the temperature observations.

[62] However, high resolution appears to degrade model performance in the PSL DJF pattern, with overestimated spatial variance, increased RMS error, and decreased correlation with observed patterns at all but the finest spatial scales. This suggests systematic errors in this simulation that McClean et al. [2011] have attributed to the intensification and contraction of the polar vortices, which consequently lead to biased planetary wave structure, evident in components 2 and 3 in our analysis.

[63] The variable that improves most in going from low to high resolution is PR. We note improvement in both seasonal climatologies in terms of improved RMS error and pattern correlation with observations, but the high-resolution model consistently overestimates the amplitude of spatial variance. Surprisingly, the high-resolution simulation has not improved the finest scale range (component 1). The errors seen in the low-resolution results at these scales (apparent in both seasons) have been enhanced in the high-resolution simulation, which implies that their origins might be in model physics issues that can not be resolved with improved resolution. Previous studies [Gent et al., 2010; Hack et al., 2006; Klein and Boyle, 2010] have shown that increased resolution improves precipitation patterns due to better resolved topography, but there is also evidence that the errors found in the coarser grid have been intensified in the finer one [Hack et al., 2006; Shaffrey et al. 2009]. McClean et al. [2011] have found that the double Pacific ITCZ present in the low-resolution simulation has been reduced, but not resolved, in the high-resolution simulation we consider here.

[64] Overall, we find that the effects of increased resolution are limited to moderate improvement in the fine-scale TAS pattern (boreal summer only) and in the PR climatologies, although the biases are enhanced at the finest scales. However, in the case of DJF PSL, high resolution appears to degrade the performance. The major shortcoming of the higher-resolution simulation is that it overestimates the spatial variability of all variables considered in comparison with low-resolution model results and observations.

[65] Similar to earlier results [Rauscher et al., 2010], we observe seasonality of the biases, with the JJA climatology being better simulated. This might indicate difficulties in simulating the DJF PSL pattern in the Northern Hemisphere due to the complexity of land-air-sea processes.

[66] Surprisingly, with the sole exception of the JJA TAS climatology, the higher-resolution simulation does not noticeably improve the representation of the finest scale range. This indicates that model performance cannot generally be improved with enhanced horizontal resolution alone.

7 Conclusions

[67] In devising these metrics for model performance at different spatial scales, we have introduced scale space analysis as a tool for model evaluation. These multiscale analysis methods, commonly used in image processing and adapted here for climate data analysis, draw upon the physics of diffusion to remove small scales from data in a systematic way. In particular, one can prove that no spurious structure is introduced into the data as a result of this “smoothing” mechanism. This enables comparisons of model and observation results at progressively larger spatial scales and yields filters that isolate variability at different scales.

[68] This paper shows that the scale space approach can be used to isolate variability at certain spatial scales, calculate the mean-square error as a function of spatial scale, and to construct filters that isolate spatial patterns at specified scales. There are many other possible applications of these techniques, since many types of analysis can benefit from considering individual scales separately. Scale space provides the basis for feature detection algorithms commonly used to detect blobs [Lindeberg, 1998; Lowe, 1999], edges [Nielsen et al., 1997; Perona and Malik, 1990], and corners [Zhang et al., 1995] in image data, and to track features through time in video applications. Additionally, the flexibility of the scale space framework allows for the construction of useful model diagnostics. For example, the correlations or lagged correlations between two variables may be probed as a function of spatial scale, giving insight into how small-scale processes affect larger-scale features.

[69] As climate models move toward increased horizontal resolution, such metrics and diagnostics will be vital in determining the costs and benefits of the resolution increase. Because global climate models incorporate physical processes on a wide variety of scales, we require a multiscale framework to analyze their performance. Scale space, based on the well-studied physics of diffusion, offers a sound basis for this analysis.

Appendix A: Low-Resolution Climatology Chunks

[70] The low-resolution model was integrated over a century-long period, but due to computing constraints only 20 years of high-resolution model output are available. Additionally, the ERA-INT reanalysis data set used spans a 20 year period, while GPCP spans a 30 year period. The climatology is therefore calculated over a much longer period in the low-resolution model, and this longer time period may bias the results by, for example, smoothing out small-scale spatial variance. However, we do not find this to be the case. Figures A1a–A1f show the Taylor diagrams of section 4 recalculated for five 20 year “chunks” of the low-resolution model. These figures show the relative insensitivity of the climatological results to the length of the time series from which they are derived. This demonstrates that the uncertainty from the internal variability does not significantly change the model climatology and particularly does not affect the conclusions about the intermodel comparison in this case.

Figure A1.

Taylor diagrams for the (top row) DJF and (bottom row) JJA climatologies of (a, b) TAS, (c, d) PSL, and (e, f) PR. The diagrams are identical to those in Figure 3, but the 100 year low-resolution simulation has been divided into five 20 year chunks, and the climatology calculated for each chunk.

Acknowledgments

[71] We thank Ken Sperber, Ben Santer, and Celine Bonfils for helpful comments. This work was supported by the Office of Science (BER), U.S. Department of Energy at Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344.

Ancillary