Evaluating a terrestrial ecosystem model with satellite information of greenness



[1] Dynamic global vegetation models (DGVMs) simulate ecosystem responses to multiple environmental factors. Traditionally, these models were evaluated with sparse ground measurements, but satellite observations now allow evaluation at model grid cell scales with near-global coverage. Agro-IBIS is the first DGVM to explicitly simulate the major crop types of the United States. We evaluate these simulated agroecosystems using satellite information of greenness for the first time at the regional scale. We compare Agro-IBIS-simulated leaf area index (LAI) and fraction of absorbed photosynthetically active radiation (FPAR) of crops, forests, and grasses with AVHRR (1982–2000) and MODIS (2000–2002) LAI and FPAR data sets across the central and eastern United States. Compared with MODIS data, Agro-IBIS overestimates growing season LAI over broadleaf crops and simulates reasonable but lower growing season LAI over deciduous forest, with an early onset bias in the southern region. Agro-IBIS underestimates needleleaf forest LAI and substantially overestimates the magnitude of grassland growing season LAI, with an early onset bias of one month. Similar bias patterns in FPAR occur in all biomes. While we do not trust the LAI/FPAR magnitudes from AVHRR data, we do suggest that AVHRR data may be used to evaluate the timing of onset and offset of the growing season of broadleaf crops and grasses. Evaluation of broadleaf crop peak LAI from Agro-IBIS and MODIS with ground measurements shows significant discrepancies. Continuing improvements in DGVMs (especially grass algorithms in Agro-IBIS) and validation of satellite data sets (especially over crops in MODIS) are needed to understand regional-scale terrestrial ecosystem processes.

1. Introduction

[2] On a global scale, agricultural ecosystems increased in area by 12% over the last 40 years and now cover nearly 40% of the land surface [Asner et al., 2004; Foley et al., 2005; Ramankutty and Foley, 1999]. The replacement of natural forests and grasslands with these agroecosystems has induced significant shifts in energy partitioning at the land surface [Betts, 2001], changes in water quantity and quality [Scanlon et al., 2007] and nutrient transport [Donner, 2003], as well as disruptions in land-atmosphere feedbacks [Avissar and Werth, 2005; Costa et al., 2007]. An understanding of the differences in response to multiple environmental factors between natural ecosystems and agroecosystems, and even among different types of agroecosystems is necessary to predict future resource availability and ecosystem functioning [Marland et al., 2003]. In order to achieve this goal, however, agroecosystems must be represented in biosphere models that can be incorporated into a global climate model framework, and these models must be validated with global data sets of environmental variables.

[3] Until recently, models have approximated managed ecosystems. When studies have needed to account for managed ecosystems, grassland vegetation characteristics have served as proxies for crops [Ramankutty et al., 2006], or the studies have relied on methods of modifying results such as the use of bookkeeping methods from observations [McGuire et al., 2001]. Currently, there are at least three examples of dynamic global vegetation models (DGVMs), including LPJlm, ORCHIDEE-STICS, and Agro-IBIS [Bondeau et al., 2007; de Noblet-Ducoudre et al., 2004; Kucharik and Brye, 2003], as well as other ecosystem models, including CLASS and GLAM [Kothavala et al., 2005; Osborne et al., 2007], that represent agroecosystems in regional modeling frameworks. These models combine the functions of biosphere models—to represent the response of biophysical and biogeochemical processes within ecosystems to climate and edaphic properties, with the ultimate goal of crop production models—the prediction of yield as it is influenced by crop type, climate forcing, and management. The incorporation of specific crop types into traditional ecosystem models allows the simulation of the growth and functioning of both natural and managed ecosystems in a consistent framework with spatially varying land cover types that respond to temporally varying climate forcing. Such a framework is necessary for the simulation of regional-scale processes such as seasonal water and nutrient transport through river networks, and land-atmosphere feedbacks such as precipitation recycling.

[4] One factor that influences energy, water, and carbon processes is phenology—the timing of critical events throughout the year. For vegetation, these critical events include budburst of perennial plants or planting and emergence of crops, the changes in structure with growth (e.g., increase in leaf area), flowering, grain fill of crops, and senescence. In addition to climate and other external forcing, the timing of these events and magnitudes of variables such as leaf area index (LAI) are functions of vegetation type. In this study, we use satellite remote sensing observations of vegetation greenness (LAI and the fraction of absorbed photosynthetically active radiation, or FPAR) to evaluate the Agro-IBIS model for natural and managed ecosystems within the central and eastern U.S. While Agro-IBIS has been extensively tested at point locations and with county-level crop yield data across a 13-state region, the model has yet to be evaluated for its ability to capture large-scale carbon and water cycles, and this is where satellite remote sensing can provide an advantage. Satellite information and county-level United States Department of Agriculture (USDA) census data have allowed the high spatial resolution mapping of the major crop regions of the U.S. [Leff et al., 2004]. Now that we know the location of croplands, as well as natural ecosystems, we can begin to use satellite information of vegetation characteristics to evaluate the ability of models like Agro-IBIS to capture the difference in growing season phenology among different ecosystems as they respond to climate.

[5] One of the challenges in evaluating model simulation results with satellite remote sensing observations of vegetation greenness is the relatively short period of record. Superimposed on this is the issue of technological issues that include the use of the same sensor on two different satellite platforms and the use of two different sensors with different data processing algorithms. Currently, the longest period of record is contained within the nearly 20-year period of NOAA/NASA Pathfinder Advanced Very High Resolution Radiometer (AVHRR) data [Justice et al., 1985], however, the more recent Moderate Resolution Imaging Spectroradiometer (MODIS) data set is more accurate [Friedl et al., 2002; Myneni et al., 2002; Tian et al., 2000; Yang et al., 2006a, 2006b]. In this study, we evaluate vegetation greenness as simulated by Agro-IBIS with data sets produced from AVHRR and MODIS observations. The overall goal is to improve our models in order to evaluate the response of carbon and water budgets to changes in climate, climate variability, and land cover or land use.

[6] Parameters within Agro-IBIS were designed to be “global” values for a particular biome. This type of vegetation model is used to simulate vegetation on a global grid, however, calibration and evaluation of model variables and processes has been limited to ground-based data sets that are not global at all but are composed of either point data or gridded data sets that are extrapolated from point measurements. We now have the ability to evaluate model simulations of regional processes (e.g., the wave of greening from south to north in spring) with satellite observations of an entire region at one instance. The goal of this study is to evaluate how well Agro-IBIS represents the length of the growing season and the magnitude of peak values of LAI and FPAR for the three major biomes of the central and eastern U.S. (i.e., cropland, forest, and grassland). In section 2 we describe the Agro-IBIS model and in section 3 we describe the AVHRR and MODIS data sets. We explain how we evaluate the simulated (Agro-IBIS) and observed (AVHRR and MODIS) variables in section 4, and describe the results of this comparison in section 5. Some suggestions for future work are given in section 6.

2. The Agro-IBIS Model

[7] The agricultural version of the Integrated BIosphere Simulator (IBIS), Agro-IBIS [Kucharik, 2003; Kucharik and Brye, 2003], includes all the components of the global IBIS model [Foley et al., 1996; Kucharik et al., 2000] along with the ability to simulate major agroecosystems of the continental U.S. (i.e., corn, soybean, spring wheat, and winter wheat). Agro-IBIS simulates vegetation canopy physics, vegetation phenology, soil physics and hydrology, and ecosystem biogeochemistry over natural and managed ecosystems (Figure 1). Much of the land surface module structure has been borrowed from the LSX land surface package [Thompson and Pollard, 1995a, 1995b], while the crop phenology routines simulate the growth and dynamic carbon allocation in a manner similar to the CERES-MAIZE and EPIC crop models [Jones and Kiniry, 1986; Sharpley and Williams, 1990]. Numerous ecosystem variables are represented, including surface system albedo, net radiation and other surface energy fluxes, optimal planting date, net primary productivity (NPP), daily LAI, root growth and turnover, soil moisture and soil water and energy fluxes, and evapotranspiration. In addition, the model simulates crop yield, dry matter production, harvest index, total plant nitrogen uptake, net nitrogen mineralization, plant tissue carbon and nitrogen, soil carbon and nitrogen, and soil carbon dioxide flux over agroecosystems. Agro-IBIS has been used to examine the effects of agricultural land cover change on the water budget, surface hydrology, and nitrate transport in the Mississippi River basin [Twine et al., 2004; Donner and Kucharik, 2003; Donner et al., 2004], and the associations between the El Niño-Southern Oscillation and patterns in Mississippi River basin water budget and surface hydrology [Twine et al., 2005]. The model is scalable from a point location to the globe, so it can be used to examine issues ranging from precision agriculture to global climate change effects on terrestrial ecosystem biogeochemistry.

Figure 1.

Schematic of the regional Agro-IBIS model that includes modules for land surface physics, vegetation phenology and dynamics, belowground biogeochemistry, and crop management. Adapted from Kucharik [2003].

[8] Many of the crop variables listed above have been validated with six years of measurements at a research station in Wisconsin [Kucharik and Brye, 2003]. Simulated maize yield has been evaluated with 37 years of USDA yield records over a 13-state region of the U.S. Corn Belt [Kucharik, 2003]. The global version of IBIS has been tested and validated against surface flux observations [Delire and Foley, 1999; El Maayar et al., 2001], global compilations of ecosystem and hydrological data [Foley et al., 1996; Kucharik et al., 2000], and regional data from the Amazon [Coe et al., 2002; Costa and Foley, 1997; Foley et al., 2002] and Mississippi River basins [Donner, 2003; Donner and Kucharik, 2003; Donner et al., 2002; Kucharik et al., 2001; Lenters et al., 2000]. Crop growth and phenology, carbon allocation, and yield were well represented in comparison to measurements from the Mead, Nebraska FLUXNET site over irrigated and rain-fed maize and rain-fed soybean managed with no tillage [Kucharik and Twine, 2007]. Simulated budburst of forests was found to have an early bias compared with observations from three sites [Kucharik et al., 2006], however, phenology of natural and managed ecosystems has not been evaluated at the regional scale before, and this is the major goal of the current study.

[9] Each 0.5° by 0.5° Agro-IBIS grid cell has the potential to represent two vegetation canopy levels: an upper level forest canopy and a lower level of shrubs, C3 (cool) and C4 (warm) grasses, and crops. The model also includes three snow layers and eleven soil layers extending to a depth of 2.5 m. Agro-IBIS explicitly represents the temperature of the soil or snow surface and the vegetation canopies, as well as the temperature and humidity within the canopy air spaces, and temperature and water content of each soil layer. Agro-IBIS simulates the exchange of both solar and infrared radiation between the atmosphere and the surface (canopy and ground). The radiative properties (e.g., reflectivity and transmissivity) of canopies, soil, and snow are uniquely calculated and then related to the incoming radiation characteristics (e.g., fraction of diffuse and direct beam radiation and zenith angle). As a result, the surface albedo is a function of the vegetation cover with prescribed leaf optical properties, the surface soil texture class and water content, and the incoming solar radiation. Leaf optical properties vary between the upper and lower canopy, and while the upper canopy (forest) properties are time invariant, the grass and crop properties vary by season depending on whether the leaves are green and growing or brown and senescent [Bonan, 1995; Sellers et al., 1996].

[10] All stages of growth of managed ecosystems (i.e., planting, leaf emergence, tasseling, harvesting) are determined by soil temperature functions and growing degree-day summations. The daily LAI of crops is determined by a combination of carbon (C) assimilation, prescribed specific leaf area, and C allocation [Kucharik, 2003]. Phenology of natural ecosystems is adapted from the method of White et al. [1997]. Onset of the growing season of natural vegetation (forest and grasslands) is determined by a threshold of growing degree days that varies by climate and vegetation type of each grid cell. Once onset of the growing season has occurred, the green LAI of natural ecosystems increases from a minimum to maximum value over a 15-day period [Kim and Wang, 2005]. Maximum LAI for any growing season is dependent on carbon assimilation from the previous calendar year. Offset of natural vegetation is determined by a combination of photoperiod (trees only), running-mean temperature, minimum temperature, and soil moisture stress (grasses only). Once offset occurs, green LAI decreases to minimum values over a 15-day period. While deciduous trees drop their leaves, grasses retain a constant LAI but leaf color changes from green to brown until onset occurs next spring, when leaves return to green. The LAI of evergreen needleleaf trees is constant for a particular year but the growing season onset and offset are still determined by climate variables. White et al. [1997] derived empirical equations for the timing of onset and offset of trees and grasses at nine locations within the U.S. based on Normalized Difference Vegetation Index (NDVI) from AVHRR data. We incorporated these algorithms and tuned the parameters such that simulated onset and offset dates best compared with the NDVI-based dates of onset and offset. This tuning was subjective and only based on phenology from the nine locations used in the White et al. study. The current study will evaluate the performance of our adapted phenology model at a continental scale over hundreds of grid cells.

[11] FPAR is calculated every time step and is dependent upon variables that vary daily such as LAI, fraction of the grid cell covered by vegetation, leaf and soil optical properties, and variables such as direct and diffuse albedo that vary on an hourly time step with solar geometry. FPAR is calculated as

equation image

where Ib (Id) is the direct (diffuse) solar radiation absorbed by the leaves of the canopy in the broadband visible band, and Sv,b (Sv,d) is the direct (diffuse) broadband visible solar radiation incident at the top of the canopy. Incident solar radiation is simulated based on data sets of daily cloud cover and hourly statistical downscaling of input data, as described below. The sum of Ib and Id, also known as photosynthetically active radiation (PAR) absorbed by the leaves (APAR), is calculated using the two-stream approximation of Dickinson [1983] and Sellers [1985] and is adapted from the LSM model [Bonan, 1996]. The APAR is dependent on Sv,b and Sv,d, LAI, leaf angle orientation, and other parameters that depend on the fraction of the grid cell that is covered by vegetation, soil, and snow. Agro-IBIS assumes a random (nonclumped) distribution of leaves for all vegetation canopies. Deciduous forests have a spherical leaf angle distribution while crops and grasses have different horizontal and vertical weighting factors used to obtain the leaf angle distribution according to crop type. Given the calculated direct and diffuse albedos of the ground, the two-stream approximation is used to calculate the direct and diffuse fluxes absorbed by the canopy, reflected by the canopy, and transmitted through the canopy for both visible and near-infrared wavebands. For comparison with the satellite data set of FPAR, which is only based on leaf absorption, the PAR that is absorbed by the leaves is then separated from total PAR (absorption by leaves and stems). Agro-IBIS FPAR is calculated on a grid cell basis (including both vegetated and nonvegetated portions of the grid cell), as for the satellite data sets.

[12] Agro-IBIS requires climate forcing for each hourly time step on a 0.5° latitude/longitude grid. We synthesized hourly weather and climate information from a combination of monthly climatic observations and daily reanalyzed meteorological data. We first used observed monthly values of air temperature, precipitation, vapor pressure, and cloud fraction from 1901–2002 along with 1961–1990 climatological mean values of monthly wind speed, diurnal temperature range, and number of wet days per month as given by the University of East Anglia Climate Research Unit data sets (CRU05) [Mitchell and Jones, 2005; New et al., 1999]. In order to capture the transient, day-to-day phenomena of storm systems characteristic of the continental U.S. and to attempt to simulate spatial patterns of weather events, we combined the monthly CRU05 data with daily anomalies of temperature, precipitation, specific humidity, and cloud fraction for 1948–2002 from the National Centers for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR) meteorological reanalysis data set [Kalnay et al., 1996; Kistler et al., 2001] to produce a daily value at each grid cell. The monthly average values of these daily values were mathematically forced to equal the monthly CRU05 values. Finally, hourly variations in climatic variables were simulated through the use of empirical formulations that relate temperature, specific humidity, precipitation, and radiation variability [Campbell and Norman, 1998]. The Agro-IBIS model was run at an hourly time step for the period 1860–2002, which allowed the vegetation biomass to reach near equilibrium by the middle of the 20th century. The maximum value of all hourly values of FPAR was chosen as the daily value. Daily FPAR and LAI were then averaged to obtain monthly values for the period 1982–2002.

3. The AVHRR and MODIS Data Sets

[13] One of the best historical data sets for vegetation monitoring is the 20-year Pathfinder AVHRR NDVI data set. The Climate and Vegetation Research Group of Boston University has provided monthly data sets of NDVI, LAI, and FPAR for the period of July 1981 through May 2001 (http://cliveg.bu.edu), and here we use the Collection 3 data sets of LAI and FPAR [Myneni et al., 1997]. These data sets can be used to drive ecosystem models that require LAI as input [Tian et al., 2004], or to evaluate the performance of models that simulate dynamic LAI, as is the case with Agro-IBIS. The algorithms exploit relationships among NDVI, LAI, and FPAR that are specific to canopy structural characteristics and their influence on radiation regimes, therefore Myneni et al. [1997] first created a land cover classification [James and Kalluri, 1994] composed of six structural classes: grasses and cereal crops, shrubs, broadleaf crops, savanna, broadleaf forests, and needleleaf forests. Then, they used the time-varying measurements of NDVI along with knowledge of the land cover and a radiative transfer model to derive LAI and FPAR at each grid cell [Knyazikhin et al., 1998; Myneni, 1991; Myneni et al., 1992].

[14] Because NDVI and FPAR have a near-linear relationship, the error in FPAR is estimated to be of the order of the uncertainty in NDVI measurements, which is approximately 20% [Myneni et al., 1997]. LAI values were validated with measurements from 73 conifer plots in 30 locations in Montana, Oregon, and California. Myneni et al. performed a series of model realizations through varying the vegetation parameters and determined an uncertainty in LAI of up to 50%; however, Buermann et al. [2002] estimated an uncertainty of only 10–20% after comparison with another LAI data set, measurements from several field campaigns, and climate data analysis. The NDVI-LAI relationship is nonlinear, and the lower sensitivity to the change in NDVI at LAI values greater than about 4 has been documented [Buermann et al., 2002; Myneni et al., 1997]. This so-called saturation problem makes evaluation of AVHRR LAI over forested regions difficult. The data sets used in the present study are 0.5° by 0.5° grid cell aggregations of these AVHRR data sets.

[15] The MODIS data sets of LAI and FPAR (MOD15A2, C4.1), also obtained from the Climate and Vegetation Research Group, provide the most current remotely sensed observations of vegetation greenness. While this data set has a shorter-time record than the AVHRR data, it is an improvement over AVHRR for a number of reasons, including different spectral information, improved calibration, cloud screening, and atmospheric correction [Friedl et al., 2002; Justice et al., 1998; Loveland et al., 1999], as well as the ability of MODIS to provide information at a range of view angles, which aids in removing noise [Lucht et al., 2000]. Surface reflectance is corrected for effects from atmospheric gases, aerosols, and thin cirrus clouds, and is then used in the LAI and FPAR algorithms [Myneni et al., 2002], eliminating the need for NDVI as in the AVHRR algorithm.

[16] The MODIS LAI and FPAR algorithms use bidirectional reflectance factors (BRFs) and look-up tables (LUTs) based on canopy structure to output the most probable value of LAI and FPAR for each 1-km pixel. The virtually unlimited variations of vegetation canopies are grouped into the same six biomes as in the Myneni et al. [1997] study, and algorithms use fixed parameters based on biome. Because the BRFs and LUTs may not provide enough information to predict a single solution of LAI or FPAR, the algorithms also use energy conservation, based on canopy transmittance and absorption, as a constraint. This step has been found to significantly improve accuracy [Tian et al., 2000; Wang et al., 2001], however, there are still many instances where this “main” algorithm fails and a “backup” algorithm (the Myneni et al. [1997] algorithm) is used. The MODIS LAI/FPAR algorithm development and evaluation are described in Yang et al. [2006b]. In this study, we use the Collection 4 LAI/FPAR monthly data sets for the years 2000–2002 that have a value of quality assurance of 50 or greater (i.e., values are derived from the main algorithm), as provided by the corresponding quality assurance data sets.

4. Evaluating Agro-IBIS With the AVHRR and MODIS Data Sets

[17] There are numerous issues involved in the comparison of results from a numerical ecosystem model with satellite remotely sensed observations of vegetation. Some issues relate to the satellite sensor observations, including calibration, geometry of the sensor viewing angle and sun angle, and atmospheric effects such as cloud contamination, and scattering or absorption effects from particles and gases [Diner et al., 1999; Justice et al., 1998]. Other satellite issues involve uncertainty in models used to derive the LAI and FPAR data sets from the reflectance observations [Myneni et al., 2002; Wang et al., 2001; Yang et al., 2006b]. Despite these uncertainties, global data sets of LAI and FPAR provide a means to evaluate vegetation greenness at the global scale on a regular, sometimes daily, basis.

[18] There is also uncertainty in the numerical simulation model, including parameter estimation, input data accuracy, and the ability of algorithms to accurately represent processes. The goal of this study is to compare simulations of monthly LAI and FPAR from the Agro-IBIS model with satellite remote sensing products of LAI and FPAR from the AVHRR and MODIS sensors. We focus on the central and eastern United States to capture the Great Plains grasslands, croplands of the Midwest, and eastern forests. Ultimately, we plan to use what we have learned in this analysis to improve the phenology of natural and managed ecosystems represented in global models.

4.1. Resolution and Land Cover Classification

[19] Two issues regarding evaluation that pertain directly to this study are (1) spatial and temporal resolution and (2) land cover classification. Both the observations and model output data sets must coincide in space and time, and some information is lost when a data set is aggregated in space and averaged over time. AVHRR LAI and FPAR data sets are provided on a 0.5° by 0.5° grid at monthly time scales, which matches Agro-IBIS spatial and temporal data. MODIS LAI and FPAR data sets are provided at the coarsest resolution of 0.25° by 0.25°, therefore we aggregated these data sets to a 0.5° by 0.5° grid by averaging the values of the four cells contained within the 0.5° grid cell.

[20] The land cover classification used to derive LAI and FPAR from AVHRR NDVI data is provided on a 16-km Goode's projection from the Climate and Vegetation Research Group. We reprojected and aggregated this data to a 0.5° by 0.5° grid by choosing the dominant land cover class within each 0.5° grid cell (Figure 2a). We aggregated similarly the 0.05° by 0.05° grid of MOD12C1Land_Cover_Type 3 (LAI/FPAR biomes) MODIS land cover classification (Figure 2b) provided by the U.S. Geological Survey Earth Resources Observation and Science (EROS) Land Processes Distributed Active Archive Center (http://edcdaac.usgs.gov/modis) that was used in the MODIS LAI and FPAR derivations.

Figure 2.

(a) AVHRR land cover classification based on Myneni et al. [1997] and reprojected and aggregated from ∼16-km Goode's projection to a 0.5° latitude/longitude grid. (b) As in (a), for MODIS. (c) Natural vegetation land cover classification as simulated by Agro-IBIS (valid 2002). Black cells are nonvegetated terrestrial cells.

[21] For natural ecosystems, land cover classification is initialized for each Agro-IBIS grid cell at the beginning of the model run using a potential vegetation (i.e., natural vegetation or vegetation that would grow in the absence of human influence) map derived from the International Geosphere Biosphere Programme's 1-km DISCover land cover data set [Loveland and Belward, 1997] and historical vegetation maps of natural ecosystem extents [Ramankutty and Foley, 1998]. The dynamic vegetation component of Agro-IBIS allows the vegetation class of each grid cell to change yearly according to climate; however, potential vegetation classes do not change significantly over the 150-year model simulation and are shown for the final year of simulation (2002) in Figure 2c. The vegetation class names in Agro-IBIS and in the satellite data sets coincide, therefore all maps in Figure 2 share the same legend. Figure 3 shows the fraction of each grid cell covered by corn, soybean, spring wheat, and winter wheat. This crop cover map, circa 1992, is used in each year of the simulation [Donner, 2003]. Agro-IBIS requires the fraction of a grid cell that contains each crop, and the remainder of the grid cell contains the potential vegetation class. Even though Agro-IBIS simulates separately the phenology for corn and soybean (and all of their other variables), we combine results for corn and soybean into a broadleaf crop category in order to compare with results from the broadleaf crop category of MODIS and AVHRR. The large fraction cover of broadleaf crops in Agro-IBIS (corn and soybean in Figures 3a and 3b) qualitatively corresponds to the broadleaf crop class in MODIS (Figure 2b).

Figure 3.

Fraction crop cover circa 1992, based on Donner [2003] for (a) corn, (b) soybean, (c) spring wheat, and (d) winter wheat.

[22] Vegetation class in the AVHRR data set is more heterogeneous than either the Agro-IBIS or MODIS classification. While the general patterns are somewhat similar, there are significant amounts of broadleaf deciduous and needleleaf tree areas within regions that should be dominated by croplands. Myneni et al. [1997] analyzed the difference in classifications between their map, derived from 8-km NDVI information, and the classification of Loveland and Belward [1997]. Myneni et al. determined that shrubs, broadleaf forest, and needleleaf forest could be successfully determined about 75% of the time. Broadleaf crops were misclassified 40% of the time as forests.

[23] Land cover mapping based on information from MODIS is considered to be of higher quality than maps created from AVHRR data for a number of reasons, most of which are similar to the improvements to LAI and FPAR products discussed above [Friedl et al., 2002; Lotsch et al., 2003]. Friedl et al. [2002] describe a method of incorporating ancillary data to improve the accuracy in classification of agricultural regions, an improvement over the AVHRR classification that is especially relevant for this study and clearly seen in Figure 2.

[24] In the following evaluation of Agro-IBIS with the satellite-derived observations, any grid cell for which the Agro-IBIS vegetation class does not agree with the satellite-derived classification is removed from the comparison. Cells where vegetation classes agree are shown in Figure 4, and the number of grid cells that agree for each comparison, along with the total number of grid cells in each classification are listed in Table 1. Agro-IBIS grid cells that contain a fraction cover of broadleaf crops of 20% or greater are evaluated with broadleaf crop grid cells from the satellite data. Because the satellite classifications do not distinguish between grasses and cereal crops, those Agro-IBIS grid cells that contain 20% or greater fraction cover of wheat are evaluated with corresponding grass and cereal crop grid cells from the satellite data and are identified in Figure 4. A significant number of MODIS broadleaf crop cells are not seen in the Agro-IBIS map over Florida and eastern Texas, and the misclassification of broadleaf crops as trees is evident in the AVHRR map across the Midwest. There are also many evergreen needleleaf forest cells in the Agro-IBIS map in Minnesota, Wisconsin, and Michigan, and in the southeastern U.S. that are not seen in the MODIS map, however, some of these are seen in the AVHRR map. Agro-IBIS does not simulate any of the savanna grid cells seen in the MODIS or AVHRR maps.

Figure 4.

Grid cells (and associated vegetation type) used in the (a) AVHRR—Agro-IBIS comparison and (b) MODIS—Agro-IBIS comparison. A grid cell is considered broadleaf crop in Agro-IBIS if 20% or more of the grid cell is covered by corn or soybean. Grid cells are designated wheat if they are grass and cereal crop according to satellite data and contain at least 20% cover of wheat according to Agro-IBIS.

Table 1. Number of Grid Cells Corresponding to Each of Five Land Cover Classifications Simulated by Agro-IBIS and Given in the MODIS (USGS-EOS, http://edcdaac.usgs.gov/modis) and AVHRR [Myneni et al., 1997] Land Cover Data Setsa
 Total Number of Agro-IBIS Grid CellsTotal Number of MODIS Grid CellsTotal Number of AVHRR Grid CellsNumber that Agree Agro-IBIS—MODIS (Percent Relative to Agro-IBIS)Number that Agree Agro-IBIS—AVHRR (Percent Relative to Agro-IBIS)
  • a

    Also shown is the number of grid cells that agree between Agro-IBIS and MODIS, between Agro-IBIS and AVHRR, and percent relative to Agro-IBIS grid cells. Although the Agro-IBIS classification varies from year to year, the values given correspond to 2002.

  • b

    Fraction cover of crop is >20%.

  • c

    Includes both grasses and cereal crops.

  • d

    Classified as grass according to satellite data set and as >20% wheat according to Agro-IBIS.

Broadleaf crops376b601496294 (78%)189 (50%)
Deciduous forest847582468276 (33%)302 (36%)
Evergreen needleleaf forest989364758352 (36%)568 (57%)
Grasses701852c724c546 (78%)483 (69%)
Wheat   151d121d

4.2. Model Simulations and Evaluation

[25] We use the monthly 0.5° by 0.5° gridded data sets of LAI and FPAR derived from AVHRR and MODIS data to evaluate the Agro-IBIS model over the central and eastern U.S. (24.5°–50°N, 105°–65.5°W) for the two time periods of 1982–2000 (for AVHRR) and 2000–2002 (for MODIS). We did not use the last five months of the AVHRR data set (January–May 2001) because it did not include the 2001 growing season. Even though the MODIS data set continues after 2002, only the 2000–2002 time period overlaps with gridded climate data needed to drive the model. Through our comparison, we hope to gain insight into the model's representation of the timing of the growing season and magnitude of greenness of both natural and managed ecosystems. Because green up or senescence of vegetation can occur within days to weeks, the monthly values are too coarse to evaluate the model's representation of phenology to better than several weeks, and this will be addressed in future work through use of the 8-day MODIS product.

5. Results

[26] In the following sections we will describe results of our analysis for (1) Broadleaf crop, (2) Forest (deciduous and needleleaf), and (3) Grass and wheat.

5.1. Broadleaf Crop Evaluation

5.1.1. LAI Magnitude

[27] In order to evaluate the timing of growing season onset and offset and LAI/FPAR magnitude at a regional scale, we averaged all LAI and FPAR values over a particular vegetation class for each month. For the broadleaf crop biome, the maximum LAI simulated by Agro-IBIS for the period 2000–2002 is 4.0, and the maximum MODIS value is 2.5 (Figure 5a). The 1982–2000 Agro-IBIS maximum is 3.6, while the maximum AVHRR value is 5.5. All three data sets show a similar minimum value of LAI less than 1.0. Agro-IBIS overestimates growing season LAI over most broadleaf crop grid cells by 0.5–2.0 compared with MODIS (Figure 6c).

Figure 5.

Monthly LAI and FPAR for (a and b) broadleaf crops, (c and d) deciduous forest, and (e and f) grasses from 1982–2000 for AVHRR—Agro-IBIS comparison and from 2000–02 for MODIS—Agro-IBIS comparison.

Figure 6.

Difference (Agro-IBIS – MODIS) in LAI (2000–02 average) by vegetation type for April, May, July, and October.

[28] Maximum broadleaf crop LAI from the MODIS Collection 3 data set (the predecessor to the data set used here) was found to be overestimated compared with ground measurements [Cohen et al., 2003; Tan et al., 2005]. This bias has been adjusted in Collection 4, however, it appears that the bias might have been overcorrected relative to Agro-IBIS output and ground measurements from the Bondville, IL, USA FLUXNET site (Table 2). There are uncertainties in comparing a grid cell value of LAI from satellite data with measurements from point locations [Cohen et al., 2003; Cohen et al., 2006; Yang et al., 2006b], however, ground estimates that agree with Agro-IBIS results suggest that MODIS may underestimate peak LAI by as much as 50% over the broadleaf crop region of the central U.S. The 2000–02 monthly values of LAI averaged over this region derived with the MODIS main algorithm are not significantly different from LAI estimates averaged from both the main and backup algorithms, however, this issue has been documented over cropland regions as a potential source of uncertainty [Cohen et al., 2006; Tan et al., 2005]. The AVHRR values are also not substantiated by the ground estimates given above and are so large that they are comparable to deciduous forest LAI; therefore we do not trust the magnitudes for our evaluation.

Table 2. Peak LAI as Simulated by Agro-IBIS, Given by MODIS LAI Data Sets (Averaged Over 2000–02) and as Reported From Ground Measurements at FLUXNET Sites for Listed Years
Land Cover and LocationAgro-IBIS (2000–02 Average Peak)MODIS (2000–02 Average Peak)Site Observation and DateReferences
Broadleaf crop Bondville, IL, USA4.9 (83% corn-soy/17% deciduous forest)1.95.0 (corn; 1997)Dr. Tilden Meyers;
 6.7 (soy; 1998)Cohen et al. [2003];
 5.5 (corn; 1999)Cohen et al. [2006];
 5.0 (soy; 2000)Meyers and Hollinger [2004];
 4.4 (corn; 2001)Wilson and Meyers [2007]
 6.0 (soy; 2002 & 2004 average)Wilson and Meyers [2007]
 4.5 (corn; 2003 & 2005 average)Wilson and Meyers [2007]
Deciduous forest    
Harvard Forest, MA, USA465.5 (1995–98 midseason average)Kucharik et al. [2006]
Deciduous forest    
Walker Branch, TN, USA5.24.8∼6 (1995–98 average peak)Kucharik et al. [2006]
Konza Prairie, KS, USA3.81.72.0–3.0 (summers 2000, 2001)Cohen et al. [2006]

5.1.2. LAI Timing

[29] Agro-IBIS and MODIS both reach peak LAI in July (Figure 5a). AVHRR increases to near peak by July but continues to increase by 0.5 in August. The Agro-IBIS–MODIS linear regression has a mean bias error of 0.41, a root mean square difference (RMSD) of 1.17, and a mean absolute percentage error (MAPD) of 7% (Figure 7a) that increases to 43% during April–October (Table 3). Because our goal is to evaluate the ability of Agro-IBIS to represent the greenness of the vegetation, we are mainly interested in the growing season LAI values; however, we present statistics of our comparison for each month, for the entire year, and for the growing season in Table 3. It is apparent from the MAPD of 7% over the entire year, that small LAI values in winter result in large negative percentage differences that offset the large positive percentage differences during the growing season. The 43% MAPD during the growing season shows that Agro-IBIS greatly overestimates broadleaf crop LAI relative to MODIS. The relatively large correlation coefficients of 0.74 in April and 0.70 in May, and of 0.63 in September and 0.72 in October suggest that Agro-IBIS is capturing the onset and offset of the growing season (Table 4). Relatively high correlation coefficients are also seen near the beginning and end of the growing season in the AVHRR regression (Table 5). This suggests that although the magnitude of AVHRR LAI may be overestimated, AVHRR LAI data may be used to evaluate the timing of the broadleaf crop growing season within DGVMs. We must keep in mind that the analysis of monthly LAI among the data sets does not capture onset differences that are less than a few weeks. Future work will make use of the 8-day MODIS product to improve the temporal resolution of onset and offset.

Figure 7.

Agro-IBIS LAI vs. MODIS LAI for all months during 2000–02 and all grid cells designated (a) broadleaf crop (294 grid cells), (b) deciduous trees (272 grid cells), and (c) grass (291 grid cells).

Table 3. Regression Statistics (Bias, MAPD, and RMSD) for the Agro-IBIS—MODIS LAI Comparison for Each of Four Vegetation Types, Each Month, the Growing Season (April–October), and the Full Yeara
 JanuaryFebruaryMarchAprilMayJuneJulyAugustSeptemberOctoberNovemberDecemberGrowing SeasonFull Year
  • a

    Also shown is the number of data pairs included in the regression (npoint)—missing values are ignored.

Broadleaf Crop
Deciduous Forest
Needleleaf Forest
Table 4. Monthly Correlation Coefficient for LAI Between Agro-IBIS Simulations and MODIS Observations Grouped by Vegetation Classa
  • a

    A grid cell is considered to be crop if MODIS land cover class is broadleaf crop and the Agro-IBIS fraction crop cover of the grid cell is greater than 20%. Forest includes deciduous trees only.

Table 5. Monthly Correlation Coefficient for LAI Between Agro-IBIS Simulations and AVHRR Observations Grouped by Vegetation Classa
  • a

    A grid cell is considered to be crop if AVHRR land cover class is broadleaf crop and the Agro-IBIS fraction crop cover of the grid cell is greater than 20%. Forest includes deciduous trees only.


5.1.3. FPAR

[30] Monthly averaged broadleaf crop FPAR follows a similar pattern as LAI with the Agro-IBIS maximum of 0.73 lying between the MODIS value of 0.64 and the AVHRR value of 0.92 (Figure 5b). The agreement between Agro-IBIS and MODIS is quite good during the growing season (April–September) with differences less than 0.15. Minimum FPAR values do not agree between observations and the model. Both AVHRR and MODIS remain between 0.3 and 0.4 throughout winter, while Agro-IBIS drops to 0.1. This is likely a result of the lack of a residue layer or other ground cover represented in Agro-IBIS after harvest [Kucharik and Twine, 2007]. Work is currently underway to incorporate crop residue into the model, which will hopefully create a more realistic simulation of the influence of a residue layer on energy, water, and carbon budgets.

5.2. Forest Evaluation

5.2.1. LAI Magnitude

[31] The maximum Agro-IBIS LAI averaged over all deciduous forest grid cells is 4.5 (for 1982–2000 and 2000–2002), which is less than the MODIS peak of 5.4 and AVHRR peak of 6.0 (Figure 5c). Compared with Agro-IBIS and MODIS, AVHRR appears to overestimate peak LAI and shows an extended growing season. Agro-IBIS LAI drops to minimum values of 0, which differs substantially from MODIS values near 1.5 and AVHRR values between 2 and 3. Agro-IBIS does allow a lower canopy beneath the forest canopy, however, these results highlight the difficulty in correctly representing dynamics (e.g., competition for light and water resources) of two canopies. In these simulations, an understory cannot successfully compete and total LAI decreases to minimum values of 0.

[32] The Agro-IBIS peak needleleaf forest LAI is 3.7 (standard deviation = 0.28), whereas the MODIS value is 4.6 (standard deviation = 0.57) in July and becomes subject to error in late fall through early spring because of observation issues at high latitudes. The range in statistics of the Agro-IBIS and MODIS comparison in Table 3 results from the range in MODIS LAI over this region from 1.4 in January to 4.6 in July, as the Agro-IBIS LAI is ∼3.7 throughout the year. Because of the MODIS issues and the fact that Agro-IBIS LAI is nearly constant year round, the difference between Agro-IBIS and MODIS is only shown for July in Figure 6i, where a consistent underestimation in LAI by Agro-IBIS is evident. There are some grid cells in the southeast U.S. classified as evergreen needleleaf forest by Agro-IBIS and AVHRR. The AVHRR values are nearly constant all year as they do not experience the observation problems found in the higher latitudes, but they are greater than Agro-IBIS values by 1–3 as found in the deciduous forest comparison.

[33] A comparison of Agro-IBIS and MODIS with ground measurements at two FLUXNET sites shows that both data sets agree fairly well, although Agro-IBIS LAI is underestimated by 1.5 and 0.8 at both sites, respectively (Table 2). Cohen et al. [2006] concluded that MODIS overestimates LAI in some forested biomes. Fang and Liang [2005] also found the MODIS product to overestimate forest LAI by 2.0–3.0 at a site in Maryland, USA. We conclude that Agro-IBIS does a reasonable job at simulating peak LAI in deciduous and needleleaf forest. If we account for the standard deviations, Agro-IBIS and MODIS LAI peak values do overlap, but because Agro-IBIS is consistently underestimated in both deciduous and needleleaf forests, model performance might be improved if peak LAI values were increased in future simulations.

5.2.2. LAI Timing

[34] The timing of onset and offset averaged over all forest grid cells coincides between Agro-IBIS and MODIS, and values are similar between data sets during the spring green up (Figure 5c) with high correlation coefficients in spring and autumn (Table 4). MODIS overshoots the Agro-IBIS peak in June but then steadily declines until both data sets agree once again in September when offset begins. The Agro-IBIS–MODIS linear regression has a mean bias of −0.92, an RMSD of 1.27, and an MAPD of −46% (Figure 7b) that improves to −13% during April–October (Table 3). These results show that growing season LAI compares quite well between Agro-IBIS and MODIS. AVHRR shows a substantially greater rate of increase in LAI between February and March, suggesting an early bias in onset, and the offset is not as sharp as in Agro-IBIS and MODIS. Because of this issue and the aforementioned saturation problem, we compare Agro-IBIS with MODIS for the remainder of the forest evaluation.

[35] The agreement in spring between Agro-IBIS and MODIS LAI averaged over all forest grid cells results from offsetting differences across the eastern deciduous forest region. Agro-IBIS simulates an early bias in the southern portion in April (Figure 6e) that moves northward into Pennsylvania by May (Figure 6f). The early bias is consistent with a previous comparison of Agro-IBIS with ground measurements from the Walker Branch forest site for 1995–98 that showed a one-month early bias in Agro-IBIS [Kucharik et al., 2006]. LAI simulated by the Common Land Model (CLM) also showed an early onset bias as well as a late offset bias throughout the eastern United States when compared with MODIS estimates from 2000, 2001, and 2003 [Kim and Wang, 2005]. In the present study, the early bias in the south is offset in the regional average by the underestimation of winter LAI relative to MODIS in the north (Figure 6e). Onset of northern deciduous forest agrees between data sets with both Agro-IBIS and MODIS reaching peaks in June. While MODIS magnitudes are greater than Agro-IBIS in October (Figure 6h), there is a clear initiation of offset by both data sets during this month. We do not see the early bias in onset from Agro-IBIS in northern forests as was found in Kucharik et al. [2006], or the late bias in offset as was found in CLM [Kim and Wang, 2005].

[36] The early bias in onset in the southern portion of the eastern deciduous forest region likely results from the model's use of a single threshold value of growing degree days to initiate onset. The southern region should realistically reach the growing degree day threshold value before the northern region, however this threshold value is apparently too low for a correct onset of the southern region. The difficulty lies in choosing a global parameter for the model that is adequate for the entire deciduous forest region. Perhaps there is a better threshold that will alleviate some of the early onset bias in the south without delaying onset in the north.

5.2.3. FPAR

[37] Agro-IBIS FPAR compares well with MODIS during the growing season (May–September), with differences of only 0.05 (Figure 5d). Agro-IBIS values range from 0.7 to 0.75 while MODIS values range from 0.75 to 0.8. Winter FPAR agreement is poor, perhaps because Agro-IBIS is not simulating perennial undergrowth in this simulation. Although FPAR is determined by LAI in Agro-IBIS and many other ecosystem models, there is no early bias of Agro-IBIS FPAR in the southern region of the domain relative to MODIS. This may result from a combination of seasonal solar geometry (external to the model and satellite derivations) and the underestimated winter FPAR in Agro-IBIS.

5.3. Grass and Wheat Evaluation

5.3.1. LAI Magnitude

[38] The maximum green LAI simulated by Agro-IBIS averaged over the grass biome is 2.7, which is substantially overestimated relative to the MODIS peak of 1.2 and AVHRR peak of 1.8 (Figure 5e). LAI is overestimated by Agro-IBIS across the grassland region (Figure 6l), and this overestimation is also consistent with ground measurements at a FLUXNET site (Table 2).

[39] Wheat phenology is difficult to evaluate because MODIS and AVHRR do not distinguish wheat from grass, and even though we know which grid cells are simulated as wheat in Agro-IBIS, the grid cells contain a mixture of wheat and grass and Agro-IBIS overestimates grass LAI. If we focus on April when most of the winter wheat in Kansas and Oklahoma is ramping up toward a peak and dominates the grid cell LAI value, we will minimize the grass bias. The average April LAI simulated by Agro-IBIS over the winter wheat region (grid cells for which winter wheat cover is >20%) is 0.6 (standard deviation = 0.5) and the average MODIS LAI is 1.0 (standard deviation 0.5). Figures 6b and 6c show the overestimation by Agro-IBIS in this region in May and July as the grass begins to grow and winter wheat is harvested. The coincidence of spring wheat maturity (∼July) with the grass growing season makes it impossible to separately evaluate spring wheat peak LAI. We can conclude from this analysis that Agro-IBIS is adequate at simulating winter wheat LAI, according to the means and standard deviations; however, more observations of grass and winter wheat LAI are needed to separately evaluate grass and wheat.

5.3.2. LAI Timing

[40] All three data sets show a similar average onset of the grass growing season with a small increase in April and then a larger rate of increase in May and June (Figure 5e); however, Agro-IBIS appears to have an early bias of onset in the south and late bias in the north (Figure 6k). Agro-IBIS simulates near peak values in June–August before a rapid decrease begins (Figure 5e). MODIS and AVHRR both peak in June, remain near peak in July, and then begin a gradual decline. The agreement between Agro-IBIS and MODIS average grass LAI in October (Figure 5e) is the result of offsetting biases across the region. Agro-IBIS shows a late bias in offset to the north and an early bias in offset to the south (Figure 6m). As with the forest phenology, this result suggests the need for improvement in the global threshold values used for grasses. The Agro-IBIS–MODIS linear regression tests produce monthly correlation coefficients that are less than those found in the crop and forest evaluations (Table 4), and the Agro-IBIS overestimation of LAI results in a mean bias error of 0.31, an RMSD of 1.06, and an MAPD of 22% (Figure 7c) that increases to 99% during April–October (Table 3).

5.3.3. FPAR

[41] Agro-IBIS FPAR during the growing season is indicative of a closed canopy with a peak value of 0.8 in July and August (Figure 5f). While total (green + brown) LAI is constant over an entire year, green LAI drops to 0 during winter, therefore Agro-IBIS simulates FPAR of 0 at this time, unlike MODIS and AVHRR. Even though MODIS and AVHRR winter FPAR values do not fall below 0.2, the large overestimation of Agro-IBIS FPAR during the grass growing season, a result of the overestimation of LAI, suggests that productivity is likely overestimated in grassland biomes in the U.S., and possibly in other grassland regions of the globe.

6. Conclusions

[42] In this study, we evaluated monthly averaged LAI and FPAR simulated by Agro-IBIS with MODIS and AVHRR products. Our evaluation shows that Agro-IBIS does a generally good job of simulating LAI and FPAR over deciduous forests and broadleaf crops of the central and eastern U.S., but the representation of grass needs improvement.

[43] Agro-IBIS simulates reasonable growing season FPAR over broadleaf crop, but LAI is overestimated compared with MODIS. Because the model simulates peak values of LAI that are similar to ground-based measurements, we would like to evaluate this biome further before making any adjustments to the model. We also suggest that MODIS may have overcorrected for a high bias in broadleaf crop LAI found in Collection 3, and is now underestimating crop maximum LAI in Collection 4. It may be feasible to use AVHRR data to evaluate the timing of onset and offset of the growing season over broadleaf crops and grasses, even though the magnitudes of AVHRR LAI and FPAR appear to be overestimated.

[44] Compared with MODIS data, Agro-IBIS simulates reasonable but slightly lower growing season LAI and FPAR over deciduous forest as a whole, but has an early bias in onset in the southern region. Peak needleleaf forest LAI is also slightly underestimated. Because of the “saturation” problem over regions of high surface reflectance in AVHRR products, we do not trust these estimates of LAI and FPAR over deciduous or evergreen needleleaf forests. The timing of onset and offset of the growing season, as well as magnitudes of LAI and FPAR from AVHRR do not agree with Agro-IBIS results or MODIS estimates over this biome.

[45] Agro-IBIS performs poorly over grasses compared with MODIS and AVHRR. The model substantially overestimates the magnitude of green LAI and FPAR over grassland and simulates a peak LAI that occurs one month later than seen in the satellite data.

[46] Limitations of our analysis are both satellite based and model based. Uncertainty in the satellite-derived data sets results from the remotely sensed nature of the data, the model algorithms used to relate observations to LAI and FPAR, and in the land cover classification. Agro-IBIS is limited by model algorithms, model parameters, and input data sets. Uncertainty in either the climate or land cover classification data sets can limit model accuracy.

[47] Currently, AVHRR provides the longest historic record of global phenology observations; however, MODIS provides a more accurate record as a result of technological improvements and advances in processing. In addition to observations of the patterns of vegetation greening, satellite remote sensing data are also being used to derive the productivity of ecosystems [Zhao et al., 2006] as well as evapotranspiration [Anderson et al., 2007], and evaluation of these products with model results will further advance the model algorithms and our understanding of ecosystem functioning. Until these products were available, modelers faced a problem similar to the one described in this study of attempting to validate global simulations of productivity and ET with scattered point measurements. The results of our analysis highlight the advantages and disadvantages of ground-based measurements, global data sets derived from satellite observations, and process-based numerical modeling. Ecosystem model validation will benefit from (1) continued validation of MODIS data, especially within crop ecosystems and at high latitudes, (2) ground-based measurements that are “scaled up” to model grid cell resolution, and (3) reprocessing of historic data sets like the AVHRR Pathfinder record to enable long-term (∼20 years) evaluation of the response of ecosystems to interannual climate variability.


[48] This research was supported by the U.S. Department of Energy's Office of Science (BER) through the Midwestern Regional Center of the National Institute for Climatic Change Research at Michigan Technological University and the South Central Regional Center of the National Institute for Global Environmental Change. The authors thank Navin Ramankutty and Holly Gibbs for help with the land cover data analysis, David Robison for help with MODIS data analysis, and two anonymous reviewers who improved the presentation of this work.