• Open Access

Validation of NARCCAP temperature data for some forest sites in the southeast United States



Validation of North American Regional Climate Change Assessment Program (NARCCAP) climate simulations is conducted for selected US Forest Service sites in the southeastern United States. Results focus on qualitative and quantitative comparisons of seasonal and monthly temperature data from NARCCAP simulations to the surface air temperature data sets developed at the University of Delaware. The Regional Climate Model, RCM3, driven by the Geophysical Fluid Dynamics Laboratory Global Climate Model underestimates average surface temperature for some areas of the southeast United States by about 4 °C for the daily average during the winter of the selected period. Copyright © 2012 Royal Meteorological Society

1. Introduction and motivation

The predicted global temperature changes (IPCC, 2007) due to the increasing concentration of atmospheric carbon dioxide and other greenhouse gases (GHGs) have prompted the US Department of Agriculture's (USDA) Forest Service to initiate assessments of the vulnerability of US forests to climate change. One of the indicators of the current warming trend is the fact that 11 of the last 12 years rank among the 12 warmest years since 1850 (IPCC, 2007). Studies show that over the next hundred years, the average temperature in the United States is expected to rise by 4–9 °F (Field et al., 2007). USDA Forest Service has deemed it necessary to initiate climate change assessment studies that may be used to formulate policies and actions that would reduce GHG emissions, initiate adaptation strategies and address the vulnerabilities of the ecological system to climate change (USDA Forest Service, 2010).

Iverson et al. (2008) explored the potential habitat for 134 eastern US tree species under six climate scenarios. Their study was based on two emission scenarios and three Global Circulation Models (GCMs) one of which was the Geophysical Fluid Dynamics Laboratory (GFDL), also used in this study. They found that climate change could have a large impact on suitable habitat for some tree species in the eastern United States, especially under higher emission scenarios. Among the species they studied, they concluded that approximately 66 species would gain and 54 species would lose at least 10% of their suitable habitat under a high emission climate change scenario. They suggested that on average most of the species habitat would move generally northeast, up to 800 km in the hottest scenario and highest emissions trajectory. Another notable conclusion was a possible retreat of the spruce-fir zone and an advance of the southern oaks and pines.

Projections such as the ones mentioned above would be undermined by uncertainties based on inaccuracies in the global modeling of spatial and temporal distribution of key climatic variables like temperature and precipitation. Even though GCMs have been used adequately in the simulation of large-scale climate patterns to statistically acceptable confidence levels, their coarse spatial resolution make them inappropriate for application to issues with a regional focus. Dynamic downscaling of these GCM products to finer spatial resolutions can arguably add value to the simulations (Castro et al., 2005; Giorgi, 2006) in some cases, especially where local factors such as topography, coastal effects, etc., exert considerable influence on the regional climate. In such cases the dynamically downscaled products will make the local heterogeneities more distinguishable and hence make them more applicable by planners in the assessment of climate change impacts on localized natural systems like forests, watersheds etc. Although forests are sensitive to several climatic variables, the temperature metrics are among the critical ones and are more solid as a validation parameter due to higher certainties in the observed values.

The North American Regional Climate Change Assessment Program (NARCCAP) climate data outputs, due to their relatively higher resolution (0.5° × 0.5°) in comparison to GCMs, offers a convenient starting point for further downscaling of climate variables for specific site applications. The ultimate aim of our project is to perform dynamical downscaling of climate simulations from GCMs starting from the relatively coarse resolutions of over 2.0° × 2.0° to finer resolutions of approximately 0.2° × 0.2°, which would be closer to the horizontal extent of some forest clusters and county administration boundaries in the southeastern United States and hence more applicable for forest resource diagnostics and predictions. A more efficient way to achieve this is to further downscale the NARCCAP data to the required resolution (0.2° × 0.2°) rather than starting from scratch (i.e. from the GCMs). The NARCCAP project is very open to this approach and encourages feedbacks from users with regard to further downscaling of their products (http://www.narccap.ucar.edu/).

As a prelude to the intended further downscaling of NARCCAP products, this paper highlights some rudimentary validation of the NARCCAP outputs from specific model combinations [GCM-RCM (Regional Climate Model)] against observations. In these combinations, the GCMs provided the initial and boundary conditions to the respective RCMs during the downscaling exercises that were performed by NARCCAP. This preliminary study aims at quantitatively verifying the recently available sample of GCM-RCM pairings from NARCAP and identifying biases relative to a well-established climatological dataset, e.g. the University of Delaware Climate data for some specific sites. Section 2 describes data and methodology, Section 3 contains major results and discussions and Section 4 highlights the conclusions. The intended further downscaling exercise and analyses will be reported in future manuscripts.

This work contributes to the broader community by providing quantitative assessments of NARCCAP products using observational data for the southeast United States. Although Wang et al. (2009) have done a more detailed evaluation of the NARCCAP outputs, they looked mostly at simulated precipitation using six RCMs and focused only on the Intermountain Region (IR) of western United States. They found that each model produced systematic biases in the central IR and the simulated winter precipitation was too large. The simulated annual cycles of precipitation were too strong while the semiannual cycles were relatively well produced.

2. Data and methodology

The NARCCAP is producing climate simulations from a set of RCMs driven by a set of GCMs over a domain covering the conterminous United States and most of Canada (Mearns et al., 2009). The RCMs are nested within the GCMs for simulations of the current period 1971–2000 and for the future period 2041–2070. The projected simulations are based on the SRES A2 emissions scenario developed by the Intergovernmental Panel of Climate Change (IPCC, 2007). The first stage of our study consists of rudimentary validation of the NARCCAP surface temperature data via spatial and temporal comparison of the current period (1981–2000) simulations with observed data from the University of Delaware (Legates and Willmott, 1990) monthly global gridded air temperature data at 0.5° resolution with the grid nodes centered on 0.25°.

The NARCCAP data used in this study were obtained from Earth System Grid (ESG) website (Mearns et al., 2007). The primary evaluation data was the 3 h surface temperature data, for the selected GCM-RCM combinations, downloaded from the ESG website. The selected GCMs namely, Geophysical Fluid Dynamics Laboratory (GFDL), Third Generation Coupled Global Climate Model (CGCM3) and the Community Climate System Model (CCSM) provided initial and boundary conditions to the selected RCMs, i.e. Regional Climate Model version 3 (RCM3), Canadian Regional Climate Model (CRCM) and PSU/NCAR mesoscale model (MM5), respectively. The selection of the GDFL-RCM3, CGCM3-CRCM and CCSM-MM5 model combinations was based on the availability of post processed and quality controlled data from the NARCAPP products. The current surface (2 m) air temperature data for the period, 1981–2000, was selected to facilitate a validation exercise with the corresponding surface air temperature (2 m reference or shelter height) from the University of Delaware dataset (Legates and Willmott, 1990).

Monthly and seasonal averages of temperature were determined for the conterminous United States region and for some selected forests sites in the southeastern United States, including Desoto, Nantahala and Ocala (Figure 1). These forests are located within or in proximity to the study area which is bounded by latitudes 30°N and 35°N and longitudes 80°W and 90°W (see box marked SE in Figure 2(b)). The central point for Desoto forest is located at lat 31°N, 88.9°W, while Ocala forest is centered at 29.1°N, 81.8°W and Nantahala is centered at 35.2°N, 83.5°W. The observed and model areal average temperature for the study area was obtained via averaging the respective gridded data within the latitudinal and longitudinal boundaries of the area. For the forest sites the grid data closest to the center of the forest was used.

Figure 1.

Selected forests in the southeast United States

Figure 2.

DJF observed temperature (a) Udel, versus simulated temperature (b) GFDL-RCM3 (c) CCSM-MM5 and (d) CGCM3-CRCM

Mean seasonal values of temperature covering continental United States from the various models and from the observations were determined and plotted spatially (only DJF is shown in Figure 2 for reasons discussed in Section 3). Model time-series of temperature data for the selected forest sites were graphically compared to the corresponding observations (Figure 3). The mean temperature for the study area (SE in Figure 1) and the mean relative bias (MRB) between the simulated and observed temperatures for each forest site were determined and presented in Tables and . The MRB was calculated as

equation image

Where TS is the simulated temperature, TO is the observed temperature and N is the total number of dataset pairs in the analysis.

Figure 3.

Time-series of simulated mean monthly temperature versus observations for (a) the three model outputs and (b) ensemble of the three models at Nantahala Forest

Daily temperature variability as simulated by GFDL-RCM3 were also compared to the corresponding NCEP-RCM3 values (Figure 6) as an additional validation tool with a higher temporal resolution. To smoothen these daily value curves, 10 day running means of the daily temperatures were used. GFDL-RCM3 was used in this comparison because it was found to underestimate (in comparison to observations and the other models) the monthly average temperature for the study region, especially in the winter season as shown in the section below.

3. Results and discussions

3.1. Spatial comparison of simulated versus observed data

The surface temperature field for selected simulations based mostly on availability of the finished products from NARCCAP was compared to observations obtained from the University of Delaware database. Qualitative validation of the seasonal mean of surface temperatures was facilitated by comparing spatial plots of the simulated and observed values (Figures 2(a)–(d)). Although our focus was on the southeast United States the spatial plots in these figures cover the whole NARCCAP domain and indicate that the biases (in comparison to observations) in DJF temperatures from specific GCM-RCM combinations might have been domain wide and not limited to the southeast. The DJF period was chosen for this discussion because the temperature time-series (Figure 3(a)) data from the various model combinations and from observations indicated that GFDL-RCM3 simulated temperature was persistently lower for this season in comparison to observations and other model outputs. The GFDL-RCM3 spatial plot also conspicuously extends colder temperatures further south (Figure 2(b)) in comparison to observations and to the other model combinations as can be seen in Figure 2(b).

3.2. Time-series plots of simulated versus observed temperature

The period 1981–1996 presented an uninterrupted monthly data from the observation as well as 3 hourly data from the different simulations for the elected forest sites. Time-series plots of observed and corresponding simulated monthly mean surface temperatures for Nantahala forest in the southeast United States for the period 1981–1996 are shown with a view to highlighting any trend or prognostic difference between the simulated data from the various model combinations and observations from the University of Delaware database (Figure 3(a)). A comparison of the ensemble mean, which was the average of the output from all three model combinations to the observations is also included (Figure 3(b)) and indicates a marked reduction in the DJF temperature bias in comparison to GFDL-RCM3 in Figure 3. The other forest sites had substantially similar plots and are excluded to reduce redundancy.

The mean for each month of the year over the period 1981–1996 for the simulated surface temperatures versus observations are shown in Figure 4(a) and a box plot indicating their maximum, minimum, median, upper and lower quartiles is included in Figure 4(b). Similar plots for the ensemble mean of the simulations versus observations are shown in Figure 5. The 10 day running mean of the GFDL-RCM3 and NCEP-RCM3 simulated daily temperature variability for Nantahala, Desoto and Ocala are shown in Figure 6. Comparisons of seasonal (DJF) mean values of observed (U-DEL) temperatures to the corresponding simulated values from various model combinations for the study area are shown in Table I. The observed mean temperature (DJF) for the selected region was 8.6 °C while the corresponding GFDL-RCM3 simulated value was 2.9 °C. Much improved accuracy was obtained from the CCSM-MM5 and CGCM3-CRCM which was 8.3 and 7.1 °C, respectively. The calculated MRBs of the simulated temperature data at Desoto, Nantahala and Ocala (Table II) also exhibited this bias in GFDL-RCM3.

Figure 4.

(a) Period (1981–1996) mean for each month of the year for the simulated surface temperatures versus observations. (b) A box plot indicating maximum, minimum, median, upper and lower quartiles for the monthly period mean—simulated and observed

Figure 5.

(a) Ensemble mean of the simulations versus observations (Udel) for the period (1981–1996). (b) A box plot of the ensemble mean of the simulations versus observations (Udel) for the period (1981–1996)

Figure 6.

Time-series of 10 day running mean of daily temperature for (a) Nantahala, (b) Desoto and (c) Ocala forest

Table I. Seasonal (DJF) mean temperatures for zone SE
Boundary conditionsRegional modelMean temperature (DJF) ( °C)
U-DEL 8.6
Table II. MRB of simulated monthly mean temperature in relation to observations (U-DEL) for the period 1981–1996
Desoto− 4.110.36− 0.59− 1.44
Nantahala− 2.671.910.56− 0.07
Ocala− 4.26− 0.29− 1.78− 2.11

The noticeably large cold bias during the DJF season for the region from the GFDL-RCM3 simulation was also evident from the time series temperature data (Figure 3(a)) at the Nantahala forest site. Comparison of Figure 3(a) and Figure 3(b) indicates that the ensemble mean of simulated mean monthly temperature is a closer estimate to the observations than the individual GCM-RCM model outputs. The same conclusion can be drawn by comparing corresponding parts of Figures 4 and 5. All of the monthly time-series data (from Figures 3–5) indicate that the cold bias tapers off as the season progresses from winter to summer time.

The cold bias is markedly reduced in the CGCM3-RCM3 simulation (mean DJF temperature of 5.1 °C for zone SE—not shown) indicating that the lateral boundary conditions from GFDL were highly enhancing the bias in the GFDL-RCM3 downscaled temperature field for this region. This seems more likely because comparison of the daily temperature values between GFDL-RCM3 and NCEP-RCM3 at each of the three forests sites corroborated this bias (Figure 6). A comparison of the seasonal (DJF) mean temperatures for NCEP-RCM3 and UDEL done by the NARCCAP team indicated a close agreement for the study area (see http://www.narccap.ucar.edu/results/ncep-results.html). Both NCEP-RCM3 and UDEL indicated a mean DJF temperature of between 8 and 10 °C.

Plausible explanations for GDFL-RCM3 results include the possibility that the coarse-scale lateral boundary conditions from GFDL could have dominated over the internal component processes of the higher resolution RCM3. In either case physical consistency of some climate variables might not have been maintained between the coarse-scale, GFDL, GCM and the higher resolution, RCM3, regional model. The ensemble mean of the three model combinations performs much better by substantially reducing the MRB in comparison to GFDL-RCM3 at all of the selected forest sites (Table II).

The differences in the dynamically downscaled simulation results from the various model combinations highlighted by this simple experiment indicates the complexities involved in selecting the most suitable GCM-RCM combination from the possible suite for a specific application or site. Forest resources, for example, are sensitive to both long-term mean values and extremes of temperature. Accurate forecasting of this field is therefore an unavoidable challenge.

The challenge is even more daunting considering that these results are based only on one emission scenario, i.e. SRES A2. This study, however, lends credence to the notion of a likely improved accuracy in the use of ensemble means in climate forecasting as opposed to reliance on any one particular model combination.

4. Conclusions and future work

The spatial and time-series display of the GFDL-RCM3 simulations from the NARCCAP products exhibited a notable cold bias during winter (DJF) in the southeast United States in comparison to observations. This rudimentary validation procedure was not capable of determining conclusively whether the significant cold bias produced by the GFDL-RCM3 simulation was mostly contributed by the driving (Global) model or by the nested RCM. In general the observed monthly mean temperature values were very well simulated by other GCM-RCM combinations.

More studies are needed to highlight the influence of the driving global model on the regional simulations, although preliminary results obtained in this study hint at the possibility that the errors arising from the imbalanced assimilation of the coarser-scale lateral boundary conditions from the GFDL climate model to the higher resolution RCM3 regional model domain could be the cause of the large cold bias in the DJF temperatures. Further analysis, e.g. the big brother experiment by Denise et al. (2002) and Herceg et al. (2006) could help eliminate the possibility that the bias is solely caused by nesting and downscaling techniques.

The study also indicated that the use of ensemble means might reduce some uncertainties from specific global-regional model matrices. Even though these preliminary results were useful in identifying cold or warm biases from the different simulations, they are not necessarily sufficient indicators of the long-term predictive capabilities of the various model combinations. However, there is often a tendency for the model errors to cascade into projected values, especially in dynamic downscaling methodology. It would therefore be prudent to always use ensemble means for any application-oriented projections. A comparison of these dynamically produced NARCCAP simulations with statistically downscaled outputs for the southeast region would be a worthy exercise. This would help to isolate the influence of dynamic downscaling technique over the results discussed herein.


This research was supported by USDA Forest Service contract AG-4568-C-08-0049 to the University of Georgia.U-DEL_AirT_Precip data provided by the NOAA/OAR/ESRL PSD, Boulder, Colorado, USA, from their Web site at http://www.esrl.noaa.gov/psd/. We wish to thank the NARCCAP for providing the data used in this paper. NARCCAP is funded by the National Science Foundation (NSF), the US Department of Energy (DoE), the National Oceanic and Atmospheric Administration (NOAA) and the US Environmental Protection Agency Office of Research and Development (EPA).