Surface radiation budgets in support of the GEWEX Continental-Scale International Project (GCIP) and the GEWEX Americas Prediction Project (GAPP), including the North American Land Data Assimilation System (NLDAS) project
 In support of the World Climate Research Program GEWEX Continental-Scale International Project (GCIP) and the GEWEX Americas Prediction Project (GAPP), real-time estimates of shortwave radiative fluxes, both at the surface and at the top of the atmosphere, are being produced operationally by the National Oceanic and Atmospheric Administration (NOAA)/National Environmental Satellite Data and Information Service using observations from GOES images. The inference scheme has been developed at the Department of Meteorology, University of Maryland, and the atmospheric and surface model input parameters are produced and provided by the NOAA/National Centers for Environmental Prediction. The radiative fluxes are being evaluated on hourly, daily, and monthly timescales using observations at about 50 stations. The satellite estimates have been found to be within acceptable limits during snow-free periods, but the difficulty in detecting clouds over snow affects the accuracy during the winter season. In what follows, this activity is discussed, and evaluation results of the derived fluxes against ground observations for time periods of 1–2 years are presented.
 Information on the spatial and temporal distribution of surface and top of the atmosphere (TOA) radiative fluxes is required for modeling the hydrologic cycle, for representing interactions and feedbacks between the atmosphere and the terrestrial biosphere [Dickinson, 1986; Henderson-Sellers, 1993; Betts et al., 1996], and for estimating global oceanic and terrestrial net primary productivity [Goward, 1989; Running et al., 1999; Platt, 1986]. Such information is also needed for validating climate models [Garrat et al., 1993; Wild et al., 1995; Wielicki et al., 2002] and for improving the understanding of transport of heat, moisture, and momentum across the surface-atmosphere interface [Berbery et al., 1999; Baumgartner and Anderson, 1999; Sui et al., 2003]. In the United States, the Interagency Committee on Earth and Environmental Sciences as well as the Intergovernmental Panel on Climate Change have identified clouds and the hydrologic cycle to be of highest scientific priority in global change research [Gates et al., 1999; Houghton et al., 2001]. Studies of long-range weather forecasting, climate, climate change, and interannual variability, are presently conducted with the aid of numerical weather prediction and general circulation models. In order to use model results with confidence, there is a need to evaluate them at scales at which they are implemented. Satellite observations are considered to be the only source of global scale information that could be used for model evaluation. Of particular interest are surface radiative fluxes, due to their dominant role as forcing functions of surface energy budgets [Wood et al., 1997; Wielicki et al., 1995; K. Mitchell et al., The multi-institution North American Land Data Assimilation System (N-LDAS): Leveraging multiple GCIP products in a real-time and retrospective distributed hydrological modeling system at continental scale, submitted to Journal of Geophysical Research, 2003, hereinafter referred to as Mitchell et al., submitted manuscript, 2003]. Information on surface radiative fluxes can also improve our understanding of physical processes associated with heat, moisture and momentum transfer across the surface/atmosphere interface, and therefore, improve model parameterizations [Chen et al., 1996]. In the last two decades, it has been demonstrated that radiative fluxes could be derived from satellite observations with reasonable accuracy [Pinker et al., 1995; Frouin and Pinker, 1995; Rossow and Zhang, 1995; Whitlock et al., 1995; Ohmura et al., 1998; Gupta et al., 1997]. Long-term satellite observations over large spatial scales are now available for implementing inference schemes for deriving radiative fluxes [Schiffer and Rossow, 1985; Rossow and Schiffer, 1999]. Among the objectives of the GEWEX Continental-Scale International Project (GCIP) [World Climate Research Programme, 1992] are (1) determination of the time/space variability of the hydrologic cycle and the energy budgets over a continental scale; (2) development and validation of macroscale hydrologic models; and (3) utilization of existing and future satellite observations for achieving these objectives.
 The GEWEX Americas Prediction Project (GAPP) is a continuation of GCIP, having the ultimate objective to develop capabilities for predicting variations in water resources, on timescales up to seasonal and interannual [National Research Council (NRC), 1998].
 NOAA/NESDIS is supporting GCIP/GAPP activities by developing new operational products from satellite observations [Leese, 1994, 1997]. This is a collaborative effort between NOAA/NESDIS, NOAA/National Centers for Environmental Prediction (NCEP), and the University of Maryland. Shortwave upwelling and downwelling (0.2–4.0 μm) radiative fluxes at the surface and at the top of the atmosphere, as derived from GOES observations, are part of this product. The inferred shortwave radiative fluxes include total and diffuse quantities (as appropriate), as well as spectral components (e.g., the photosynthetically active radiation (PAR)). The interface between the satellite data and the inference models has been developed at NOAA/NESDIS [Tarpley et al., 1996]. NOAA/NCEP provides information on the state of the atmosphere and surface conditions, as available from the analyzed output fields from the Eta model [Rogers et al., 1996]. The University of Maryland is involved in model development and modifications [Pinker and Laszlo, 1992a, 1992b; Pinker et al., 2002], sensitivity studies, validation against ground observations, data archiving, and data distribution. The Surface Radiation Budget (SRB) model is implemented at NOAA/NESDIS in real time on an hourly basis, for 0.5° targets for an area bounded by 66°–126°W longitude and 24°–54°N latitude belts. For each target, at appropriate forecast times, selected data from the NCEP regional forecast model are delivered to the satellite data stream, as inputs to the SRB model. This approach ensures timely and high-quality information input to the satellite inference scheme. In turn, the derived radiative fluxes help to diagnose the NCEP forecast model as to its ability to predict correctly radiative fluxes. Initial evaluation of the SRB product was done at several levels, such as running the model operationally and off line for same cases, as well as evaluation against ground truth, as available from independent projects such as the Baseline Surface Radiation Network (BSRN) [Ohmura et al., 1998; Whitlock et al., 1995], the Surface Radiation Monitoring Network (SURFRAD) [Hicks et al., 1996; Augustine et al., 2000], by independent investigators, and within the framework of the North American Land Data Assimilation System (N-LDAS) activity [Cosgrove et al., 2003a; Luo et al., 2003].
2. Current Status of Data Availability
 The first version of the experimental real-time product has been available since January 1996. All the input and output parameters, produced in support of GCIP/GAPP (currently, a total of 71), are stored at the University of Maryland, where the SRB data are evaluated against ground observations, partially quality controlled, and prepared for distribution via the World Wide Web and an anonymous ftp site, as described at http://www.atmos.umd.edu/~srb/gcip/webgcip.htm. Four types of information are archived: (1) satellite based information, used to drive the model; (2) auxiliary data used to drive the model; (3) Eta model output products relevant for hydrologic modeling; and (4) independently derived satellite products.
 Examples of surface shortwave radiation fields at various timescales, as well as examples of by products, such as net absorbed radiation at the surface and within the atmosphere, cloud amounts and cloud optical depths, are illustrated in Figure 1. Information on instantaneous, daily, monthly mean surface and TOA shortwave and photosynthetically active radiative fluxes are provided at the University of Maryland World Wide Web site almost in real time.
3. The Inference Model
3.1. General Features
 A modified version of the GEWEX SRB algorithm [Pinker and Laszlo, 1992a; Whitlock et al., 1995; Ohmura et al., 1998] (version 1.1), developed at the University of Maryland, is used. The algorithm estimates downward and upward fluxes both at the top of the atmosphere (TOA) and at the surface. A diagram of the flux retrieval process is presented in Figure 2. The TOA downward flux (Ftd) is calculated from the extraterrestrial solar spectrum by accounting for the variation in Sun-Earth distance and the position of the Sun in the sky relative to the local vertical (solar zenith angle). The downward flux at the surface (Fsd) is obtained by determining what fraction of Ftd reaches the surface as the radiation is transferred through the atmosphere. This fraction, which is referred to as the flux transmittance (T), depends on the composition of the atmosphere (e.g., amount of water vapor and ozone, optical thickness of cloud and aerosol), on the length of the path the radiation travels through the atmosphere (determined by the solar zenith angle), and to a lesser degree, on the albedo of the surface. Once T is known, the surface downward flux is obtained as Fsd = TFtd. The algorithm estimates T from the satellite-derived TOA albedo (as described below). This is possible because for a given atmosphere and surface, the TOA albedo and the flux transmittance are uniquely related to each other. Once Fsd is known, the upward flux at the surface (Fsu) is calculated as Fsu = AsFsd, where As is the surface albedo. Similarly, the flux reflected to space by the Earth-atmosphere system (TOA upward flux, Ftu) is obtained from the product of Ftd and the TOA albedo (At), namely, Ftu = AtFtd.
T is determined from a comparison of modeled values of the shortwave (0.2–4.0 μm) TOA albedos to the shortwave TOA albedo obtained from the satellite measurement, and the transmittance corresponding to the modeled TOA albedo that matches the satellite-derived value is selected. For practical reasons, the pairs of albedos and transmittances are calculated for atmospheres with a nonreflecting lower boundary. The surface reflection is added in a separate step. The modeled TOA albedos and the corresponding transmittances are calculated at five spectral intervals (0.2–0.4, 0.4–0.5, 0.5–0.6, 0.6–0.7, and 0.7–4.0 μm) for discrete values of the solar zenith angle, amount of water vapor and ozone, aerosol and cloud optical thickness, using the delta-Eddington radiative transfer method described in Joseph et al. . Radiative properties of aerosols and clouds are taken from the Standard Radiation Atmospheres [WCP-55, 1983] and from Stephens et al. , respectively. Absorption by ozone and water vapor are parameterized following Lacis and Hansen . The albedo-transmittance pairs are made available in a lookup table for the algorithm separately for clear and cloudy atmospheres, and the flux transmittances for clear and cloudy skies are determined by matching the satellite-observed clear and cloudy shortwave TOA albedos, respectively. For a given solar zenith angle, surface albedo and amount of ozone and water vapor, the matching process involves the adjustment of the aerosol optical depth for clear sky and that of the cloud optical depth for cloudy sky.
 For GCIP/GAPP, the satellite-observed TOA shortwave albedo is obtained from the visible (0.55–0.75 μm) radiance measured by the imager instrument onboard the GOES 8 satellite, using the NOAA/NESDIS prelaunch calibration [Weinreb et al., 1999] and spectral and angular transformations [Zhou et al., 1996] (for details see section 3.2). In deriving the fluxes, first the surface albedo is estimated from the “clearest” shortwave TOA albedo observed over a number of days (clear-sky composite albedo), and then corrected for Rayleigh scattering, aerosol extinction, and absorption by ozone and water vapor. In this step, the amount of aerosol is specified according to the Standard Radiation Atmospheres [WCP-55, 1983]. For GCIP/GAPP, the column amount of ozone is taken from the McClatchy atmospheres (F. X. Kneizys et al., unpublished data, 1988) as a function of latitude and season, while water vapor is from the NCEP Eta model. Next, albedo-transmittance pairs are selected from the lookup table according to the solar zenith angle, water vapor and ozone amount, and are combined with the surface albedo to yield shortwave TOA albedos. One set of pairs is for varying values of aerosol optical depth (clear atmosphere), and the other is for varying values of cloud optical depth (cloudy atmosphere). Finally, the shortwave albedos derived from the instantaneous satellite-observed clear-sky and cloudy-sky radiances are matched with the clear and cloudy sets of albedo-transmittance pairs, and clear-sky and cloudy-sky transmittances, and from these, clear-sky and cloud-sky fluxes are obtained. The clear-sky and cloudy-sky fluxes are then weighted according to the cloud cover (defined as the ratio of number of cloudy pixels to the total number of pixels) to get the all-sky fluxes. The implementation of the model requires preprocessing of the satellite data and separation of clear and cloudy radiances. This is performed at NOAA/NESDIS [Tarpley et al., 1996], as described in section 4.
 The various elements of the GEWEX/SRB algorithm have been tested in a number of different ways. The radiative transfer component has been evaluated in the framework of the Intercomparison of Radiation Codes in Climate Models (ICRCCM). The results were found to be in good agreement with those from high-resolution radiative transfer models [Fouquart et al., 1991]. The differences in atmospheric absorption, when compared to high-resolution computations (namely, standard deviations of differences expressed as percentage of the average absorption for the reference (high resolution) model) are about ±2% and ±7% for the clear and cloudy cases, respectively. Surface down-flux estimates have been compared with values measured at several locations and in the framework of various activities, such as the Satellite Algorithm Intercomparison sponsored by WCRP and NASA [Whitlock et al., 1995], reporting agreement with ground observations within 10 Wm-2 on a monthly timescale.
 In the SRB algorithm, the fluxes are calculated in the spectral intervals of 0.2–0.4, 0.4–0.5, 0.5–0.6, 0.6–0.7 and 0.7–4.0 μm. Thus it is possible to obtain fluxes at spectral intervals known to be of significance (e.g., photosynthetically active radiation). This is important, because current GCMs are run in a mode that separates shortwave fluxes at 0.7 μm [Roesch et al., 2002], to allow incorporation of newly derived satellite based parameters, such as fractional vegetation cover, derived from the Normal Difference Vegetation Index (NDVI), and for improving parameterizations of surface/atmosphere interactions [Gallo and Huang, 1998; Goward and Huemmrich, 1992; Townshend and Justice, 1995; Gutman et al., 1995]. Moreover, shortwave fluxes are separated into direct and diffuse components, which is of interest for improved modeling of radiative interaction with vegetation and oceans. Other parameters that are derived include clear-sky and all-sky albedos at the top of the atmosphere and at the surface, and aerosol and cloud optical depths. The shortwave and spectral fluxes are computed separately for clear and all-sky conditions, thus making it possible to derive information on the radiative effects of clouds, known as “cloud radiative forcing” [Ramanathan et al., 1989].
3.2. Modifications for GOES 8
 Prior to the operational implementation of the SRB model at NOAA/NESDIS, spectral/angular corrections appropriate for the filter functions of GOES 8 [Menzel and Purdom, 1994] were developed. The characteristics of the GOES 8 satellite channels are presented in Table 1. The visible channel on the GOES 8 satellite is a narrowband channel. Derivation of broadband fluxes requires development of narrow to broadband transformations for this satellite. A study was undertaken to obtain angularly dependent relationships between the broadband reflectance and the narrowband reflectance, as observed from the visible channel of GOES 8 (0.52–0.72 μm) [Zhou et al., 1996]. Top of atmosphere broadband and filtered narrowband reflectances (representing the visible channel of GOES 8) were simulated using the LOWTRAN 7 radiation code [Berk et al., 1987] for various viewing and solar geometries (represented by angular bins) and taking into consideration different amounts of water vapor, ozone, aerosol and clouds (Figure 3). Surface albedo was also varied according to the spectrally and angularly dependent models of Briegleb et al. , as representative of four broad surface types: ocean, vegetation, desert and snow. For each surface type, two sets of transformation coefficients were derived by performing linear least squares regressions between the simulated narrowband and broadband reflectances. One set of coefficient was obtained from regressions for individual angular bins, while the other was derived using all angular bins. The difference in the broadband albedo obtained from the two sets of transformation coefficients depended on the solar zenith angle and the narrowband reflectance. Large solar angles and bright scenes produced large differences of the broadband albedo. For narrowband reflectances less than about 0.15, the broadband albedos derived from the two sets of transformations generally differed by less than 1%. In the current version of the SRB model, as implemented by NOAA/NESDIS, the transformation based on the ensemble of simulations has been used.
 The process of inferring surface radiative fluxes from the satellite observations involves: interception of the real-time satellite data at NOAA/NESDIS; building transfer capabilities between the Eta model products and the satellite data; building an interface between the satellite data and the inference model; and executing the inference scheme (Figure 4). In what follows, each activity will be described independently.
4.1. Real-Time Operation at NOAA
 In Table 2, a list of quantities required as input to the algorithm is presented. In Table 3, the algorithm outputs are listed. The satellite input to the model for each grid cell is clear sky radiance, composite clear sky radiance (this is the “clearest radiance” seen during several days), cloudy radiance, and fractional cloud cover over the selected grid. Total precipitable water and, if available, the total ozone are accessed from the NCEP Eta weather forecast model. The default option for cases for which such information is not available is the use of climatological values.
Table 2. Quantities Required by the GCIP/SRB Model
calculate fluxes for clear-sky conditions
calculate fluxes for cloudy conditions
Composite clear radiance
calculate surface albedo
Target fractional cloud cover
weight clear and cloudy fluxes
Water vapor and ozone amount
select atmospheric transmittance
weight snow-free surface albedo and snow albedo
Solar and satellite zenith angles, Sun-satellite relative azimuth angle
Target latitude and longitude
scene type, surface albedo model, and climatological aerosol loading
Table 3. Output Parameters From the GCIP SRB Model
 The GOES GCIP products are derived hourly for targets that are centered at 0.5° latitude/longitude intervals. The targets consist of 9 × 8 arrays of 4 km center-to-center pixels (at nadir), where the visible data have been averaged to 4 km resolution, to coincide with the 4 km resolution infrared pixels. The methodology for deriving quantities that are directly computed from the hourly GOES data is based on the one used at NOAA/NESDIS in the early stages of insolation estimates [Tarpley, 1979; Justus et al., 1986]. The cloud detection algorithm is based on the premise that as seen from geostationary orbit at each location for a given hour, the clear-sky surface brightness changes very slowly in time. Changes that occur are caused by seasonal change in solar illumination geometry, and by changes in surface albedo, caused by seasonal variations in vegetation and surface conditions. The algorithm assumes that any partial cloud cover in a target increases the variance in the visible and infrared radiances over the target.
 The clear composite radiance (CCR), an input to the model, serves as a threshold in the cloud detection algorithm. It represents the target radiances under clear sky conditions with minimum aerosol and atmospheric contribution to the signal. The fields of CCR are maintained for every target at every daylight hour for which the solar zenith angle is less than 75°. Updating of the CCR fields is done very conservatively, to avoid cloud contamination. The CCR fields are maintained for the visible data (channel 1). Corresponding fields of clear sky visible standard deviations, σ1, and thermal standard deviations (channel 4), σ4, are also maintained.
 For a specific target at a given observation time, the target is updated if it is certain that there are no clouds in the target. To insure completely clear conditions, a target is updated if the following criteria are met: (1) there is no snow cover; (2) A1 < + k0 (3) σ1 < k1 and (4) σ4 < k2 where A1, σ1, and σ4 are the currently observed mean albedo and channel 1 and 4 standard deviations respectively, and and are the clear sky values in the CCR fields. The value of k0, k1, and k2 are specific for each target, and currently, are set as 1.5, 4.0, and 4.0, respectively. When the observation from a target meets the above criteria, the CCR is updated with a weighted average of the new and the old values, the new being given a weight of 0.25. The coefficients are strictly selected to prevent subpixel cloud cover from being included in the radiances. This conservative approach prevents cloud contamination from slowly brightening the CCR fields. If specific targets in the CCR field do not require an update for a prolonged period of time, the above criteria are relaxed, until new values are obtained.
4.3. Cloud Detection
 The values of and are used to define thresholds between clear and partly cloudy pixels in the cloud detection algorithm. Each pixel in each target is classified by its top of the atmosphere albedo as being clear, partly cloudy, or cloudy, according to the following criteria:
where k3 is currently set as 9.0 everywhere. These criteria were chosen by taking images of different scenes and calculating the cloud amounts with varying thresholds, until the cloud amounts matched the visual observations, particularly, for optically thick clouds that have a large effect on insolation. For each category of pixel in the target (clear, mixed, and cloudy), mean albedo and mean brightness temperatures in each thermal band are calculated, and the number of pixels in each is counted. The mixed pixel count is redistributed into the clear and cloudy counts, by weighting the number of mixed pixels by the mean albedo of the mixed pixels, according to the following procedure:
Amixed is the mean albedo of the mixed pixels, and
is the brightness threshold between clear and mixed pixels. The final number of clear and cloudy pixels, n and n′, respectively, are converted to clear and cloudy fractions dividing by 72, the number of pixels in the target. The clear and cloudy sky albedos, the cloudy and clear fractions, and the composite clear albedo are then provided to the insolation algorithm, along with target snow cover, and the total precipitable water. A new NOAA/NESDIS Interactive Multisensor Snow and Ice Mapping System (IMS) product [Ramsay, 1998] replaced Air Force snow information in 1999. The cloud detection method described here is successful in identifying optically thick cloud cover, but it is less reliable for detecting thin cirrus clouds. However, thin cirrus clouds have much less effect on insolation, which is the primary quantity that is being retrieved. The total precipitable water is from 6 and 12 hour Eta model forecast fields, interpolated to the target location and observation time.
5. Results and Discussion
 About 50 ground observing stations are used to evaluate the satellite estimates of the surface shortwave fluxes. These include four Surface Radiation (SURFRAD) stations (Goodwin Creek, MS; Boulder, CO; Bondville, Il; Fort Peck, MT) [Hicks et al., 1996], 21 stations from the Arizona Meteorological Network (AZMET) [Brown, 1989], about 20 stations from the Illinois State Water Survey [Hollinger et al., 1994] and about 10 stations from the Integrated Surface Irradiance Study (ISIS) [Hicks et al., 1996]. ISIS provides basic surface radiation data with repeatability, consistency, and accuracy based on reference standards maintained at levels better than 1% to address questions of spatial distributions and time trends, at sites selected to be (1) regionally representative, (2) long-term continuous, and (3) strategic foci. It should be noted that the SURFRAD network is also part of ISIS, which operates at two levels: ISIS level 1 monitors incoming radiation only, and level 2 (SURFRAD) focuses on surface radiation budgets. The stations used in the validation effort were unevenly distributed over the United States or within the respective states as shown for Illinois and Arizona in Figure 5. The evaluation against ground observations is done at the following timescales: (1) hourly; (2) daily; and (3) monthly. At the SURFRAD sites, data of high temporal resolution are available. The 3 min observations are being used in experiments to evaluate time/space compatibility between ground and satellite observations, to be discussed in section 6.4. The quality of all model input parameters is not fully known. While there is confidence in the precipitable water information, as discussed and evaluated in Yarosh et al. , less confidence can be placed on snow information. The ground observations of SW radiative fluxes are accepted as provided by each investigator, who follow calibration and quality control procedures, as described in each relevant publication. There is no cross calibration between the various networks. Some of the results presented, include all available ground truth, as well as all available satellite observations, while others include preliminary quality controlled data, such as eliminating observations outside a 3σ limit. The options followed will be specified in each case.
5.1. Evaluation of Results on Hourly Timescale
 In Figures 6a–6d, results of evaluation against ground observations from Arizona, Illinois, SURFRAD, and ISIS stations, respectively, on hourly timescales, are illustrated. Owing to the large amount of data, Figures 6a–6d include data points from one month only. Information on root mean square (rms) errors, bias, correlation coefficients, and the total number of available observations is also given. Similar computations were performed for 11 months of 1996 for all the stations, except for ISIS, for which only five months of data have been processed. December was not included in the statistics because satellite observations were available only for about two weeks. (Comprehensive summaries at hourly timescale for each month of 1996, for each of the four groups of stations are presented in Figure 10). All data used in the evaluation were subjected to preliminary quality control by eliminating cases outside 3σ from the mean. Experiments have indicated that for most cases, such a limit is below 300 Wm−2. To simplify computations, the elimination criteria were set to that latter limit. The average percentage of points eliminated from the analysis under the 3σ limit was 1.7% for Arizona, 8.5% for Illinois (for snow free conditions), 6.3% for the SURFRAD, and 4.9% for ISIS.
 The average rms at hourly timescale for all the months used in the analysis for Arizona, Illinois, SURFRAD, and ISIS stations were 87, 100, 104 and 95 Wm−2, respectively. The corresponding biases were −29, −36, −18, and 5 Wm−2, respectively. Were we to remove the winter months from the statistics for the Illinois and the SURFRAD stations (that include stations in higher latitudes), the rms for these stations would be reduced to 83 and 90 Wm−2, and the bias to −10 and −5 Wm−2, respectively. The larger errors for the Illinois and SURFRAD stations found during the winter months are believed to be due to errors in snow cover information and cloud detection over snow. The reasons for the differences between the satellite based fluxes and the observations can be due to changes in calibration of the satellite sensor, deficiencies in cloud screening methods, incorrect information on aerosol optical depths, inherent differences between the compared variables, missing satellite observations during certain hours of the day, and errors in ground observations. It should be noted that in subsequent years, the number of missing satellite data during the course of the day was reduced. Preliminary investigations indicate that correcting for possible degradation of the visible sensor could substantially reduce the bias.
5.2. Evaluation of Results on Daily Timescale
 In Figures 7a–7d results of evaluation against ground observations from Arizona, Illinois, the SURFRAD and the ISIS stations, respectively, on daily timescales, are presented. Owing to the large number of data points only four months for each station have been included in the Figures 7a–7d. For Arizona, data are included for January, May, June, and November. For Illinois and the SURFRAD stations, data from January, May, August, and November were used. For the ISIS stations, data from May–September were used. (The rms error, bias, and correlation coefficient, were not computed for this subset of data). (Comprehensive summaries at daily timescale for each month of 1996, for each of the four groups of stations are presented in Figure 10). The average rms at daily timescale for all the months used in the analysis for Arizona, Illinois, SURFRAD, and ISIS stations were 22, 26, 29 Wm−2, and 26, respectively. The corresponding biases were −7, −11, −3 and 8 Wm−2, respectively. Were we to remove the winter months from the statistics for the Illinois and the SURFRAD stations, the rms error at these locations would have been reduced to 21 and 26 Wm−2, respectively.
5.3. Evaluation of Results on Monthly Timescale
 In Figures 8a–8d results of evaluation against ground observations from Arizona, Illinois, the SURFRAD and the ISIS stations, respectively, on monthly timescales, are presented, using the consolidated data base. The monthly mean rms values obtained for Arizona, Illinois, SURFRAD and the ISIS stations are 14, 19, 15 and 15 Wm−2, respectively. The corresponding biases are −8, −11, 1, and 8 Wm−2, and the correlation coefficients are 0.99, 0.97, 0.98 and 0.97, respectively. Were the snow months eliminated from the statistics over Illinois and SURFRAD locations, the rms error would have been reduced to 9 and 11 Wm−2, respectively. In Figure 9, the diurnal variations of the shortwave fluxes at the SURFRAD stations for May 1996 are illustrated. These results are encouraging, because until now, information on radiative fluxes that resolves the diurnal cycle was not available.
 In Figures 6a–6d and Figures 7a–7d, illustrated are results for selected time periods only. Comprehensive summaries for each of the four groups of stations at hourly and daily timescales for each month of 1996 are presented in Figure 10. Corresponding correlation coefficients are presented in Figure 11.
5.4. Evaluation Experiments at High Temporal Resolution
 The observations at the SURFRAD stations are made at high temporal frequency and are averaged at 3 min time intervals (at the other stations, only hourly averages are available). This allowed to perform the following experiments over the SURFRAD stations at Bondville, Illinois: (1) ground observations were averaged over one hour, centered at the time of the satellite overpass, and satellite based fluxes were averaged over 3, 9, 15, and 60 min; (2) ground observations were averaged over 3, 9, 15, and 60 min, and satellite based fluxes were instantaneous values. In Figure 12 presented are results for the second experiment. The best results were obtained when both the satellite and ground observations were averaged over a 1 hour interval. It is reasonable to expect that the optimal time averaging depends on the average speed of cloud movement in, and that at other locations and different seasons, other optimal averaging periods would be appropriate.
5.5. Special Results
 For some locations, data from several sources were available. For example, at Bondville, Illinois, observations are being made by the Illinois Water Survey and by SURFRAD, at a distance of several meters from each other. Preliminary comparisons between these two stations for several months indicated discrepancies as illustrated in Figure 13a. After discussions with site scientists, the sources of such discrepancies were found, and corrected (Appendix A). Similarly, at Tucson, AZ, data were available from the AZMET network (32.17°N; 110.57°W; 713 m elevation), as well as from the University of Arizona (W. D. Sellers, personal communication, 1996). The differences between these two stations can be quite high, as illustrated in Figure 13b. The causes of these differences are not fully known. In Figure 14, we present the frequency distribution of the hourly ground observations (G) (upper panel) and satellite observations (S) (middle panel), in intervals of 40 W m−2, using data from all the stations in Arizona (a), Illinois (b), SURFRAD (c) and ISIS (d) for time periods up to two years. The difference between the satellite estimates and the ground observations is illustrated in the lower panel. As evident, this difference is in the range between 0 to 5%, indicating that both types of observations represent a similar radiation climate.
5.6. Estimates of Accuracies of Surface Observations
 In the framework of the Atmospheric Radiation Measurement (ARM) activity [Stokes and Schwartz, 1994], radiation measurements are being made at the Oklahoma Central Facility site located near Billings, Oklahoma using three sets of independent radiometers. These are operated and maintained per BSRN specifications [Michalsky et al., 1999]. The instruments used include an Eppley PSP, 8-48 Black and White, NIP, PIR, a SCI-TEC (now Kipp and Zonen) tracker; they are calibrated once per year and cleaned daily. Shi and Long  used such observations for five years (1997–2001) to develop estimation criteria. Calculated are mean and average absolute deviations of the yearly 95% level values and defined are yearly “best,” “typical,” and “worst” disagreements. The operational limits are presented in Table 4 and are reported in units of Wm−2. Most instruments are not operated under “best” conditions, and therefore, these documented errors need to be considered when interpreting the evaluation of the satellite estimates against the ground observations.
Table 4. Operational Limits on Measured Short-Wave Radiationa
 Estimates of surface radiative fluxes from GOES satellite observations both SW and PAR, as well as albedos, have been used in various investigations, spanning a broad range of topics, such as evaluation of mesoscale models [Berbery et al., 1999], computations of surface fluxes over the Atlantic Bight [Baumgartner and Anderson, 1999], evaluation of crop production models (K. Kunkel, personal communication, 2001), and snow melt [Cline and Carroll, 1998]. Examples will be presented in what follows.
6.1. Assessments of Operational NWP and Hydrologic Models
 Assessment of land surface energy budgets, from regional and global models has been performed by Berbery et al.  for one summer month (August 1997) and for one winter month (January 1998). The operational models are: the Eta model, the Mesoscale Analysis and Prediction System (MAPS) and the Global Environmental Multiscale (GEM). The National Center for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR) Reanalysis model outputs were used in the intercomparison. Surface energy budgets, derived from various models, were compared against ground truth, using data from the Southern Great Plains Region and the NOAA/NESDIS satellite estimates were used to evaluate the shortwave surface radiation. It was shown that the shortwave radiation biases of the models were about 25–50 Wm−2. Daily time series showed that model estimates tended to miss the amplitude of the day-to-day variability. It is believed that this is due to difficulties in parameterizing the total cloud cover. Results of Berbery et al.  showed that the previous adjustments to the radiation scheme in the Eta model still require further corrections; similarly, their results prompted improvements to the parameterizations in MAPS, resulting in improved performance. Under GCIP it was aimed to assess the accuracy at which water and energy budgets can be “closed” on a continental scale [NRC, 1998]. The Mississippi River Basin was chosen as a site for a first continental scale experiment because the Mississippi River Basin is one of the major river systems of the world and it has a very dense observational infrastructure and is rich in historic meteorological data. Results summarizing the findings on the Water and Energy Budget Synthesis (WEBS) in the Mississippi River Basin are presented in Roads et al. . Information on radiative fluxes as derived from satellite observations was used in the evaluation process.
6.2. Assessments of Research Models
 Climate and numerical weather prediction models have advanced to fairly sophisticated land surface process schemes for partitioning the energy and moisture fluxes at the surface. These schemes are responsible for maintaining soil moisture fields for model initialization, and for use in estimating runoff. The schemes are normally driven by the model estimates of precipitation, radiation, cloud cover and air temperature and humidity. Errors and bias in model forecasts of these quantities cause the soil moisture value to drift away from reality. Uncoupled surface models are being tested in Land Data Assimilation Schemes (LDAS) [Cosgrove et al., 2003b; Mitchell et al., submitted manuscript, 2003] in which surface models are forced as much as possible with real observations instead of model output. Satellite observations of cloud cover and insolation and rain gage and radar observations of precipitation are used to force the uncoupled models. The assumption being that the real observations are not affected by the NWP model biases and therefore will maintain better soil moisture fields that can be used to initialize NWP models. Examples of such applications are given in Cosgrove et al. [2003b] and Luo et al. .
6.3. Net Primary Productivity
 At the Midwestern Climate Center, Illinois State Water Survey, work is in progress on estimating operationally corn and soybean yield estimates over large areas. The area of study is the nine-state region of Illinois, Indiana, Iowa, Kentucky, Michigan, Minnesota, Missouri, Ohio, and Wisconsin that account for about 70% of the United States production of both corn and soybean. The yield estimation models [e.g., Jones and Kiniry, 1986; Wilkerson et al., 1983] require a set of variety coefficients, soil characteristics, daily weather data such as precipitation, maximum and minimum temperature, solar radiation (from which the photosynthetically active radiation (PAR) is estimated), planting dates, depth, and density. Until now, solar radiation was estimated from such parameters as cloud cover. The surface observing cloud networks are being replaced with the Automated Surface Observing System (ASOS) stations (V. L. Nadolski, unpublished data, 1998), which do not measure cloud cover above 12,000 ft. Therefore there is a concern about the effect this might have on the estimation of solar radiation by conventional methods. An experiment was conducted (K. Kunkel, personal communication, 2001), to derive corn yields for the 1996 growing season, once using conventionally derived information on PAR which enters the model, and once, satellite-derived estimates as produced for GCIP. The model yields were generated for 87 crop reporting districts in the central United States, where each data point represents a value for a single crop reporting district. These preliminary results indicate that replacing the current sources of information on surface radiative fluxes with satellite inferred values, is a viable option.
6.4. Oceanic Applications
 In middle to low latitudes, the net shortwave radiation is the largest daytime component of the net air-sea heat flux [Baumgartner and Anderson, 1999]. Errors in this component will propagate into estimates of net heat fluxes. Currently, most coastal ocean models rely on surface fields as available from Numerical Weather Prediction models for surface boundary conditions. Using buoy observations made recently during a one year Coastal Mixing and Optics (CMO) Experiment at a midshelf location south of Cape Cod, Massachusetts, surface flux fields as available from three regional Numerical Weather Prediction models that were in operation during 1996 and 1997 at the U.S. National Centers for Environmental Prediction (NCEP), were evaluated against the observations. The models used were: the Eta-48, Eta-29, and the Rapid Update Cycle (RUC-1). It was found that the Eta-29 and RUC-1 models overestimated the net ocean-to-atmosphere heat flux by about 83 and 74 Wm-2. Experiments were conducted for July–September 1996, where the shortwave fluxes as produced in support of GCIP were used. It was found that the use of such fluxes in the CMO flux fields reduced the variability in the net heat flux errors by 46%. The correlation coefficients between the observations and the flux fields with and without the GCIP shortwave information over the same period were 0.929 and 0.863, respectively.
 In this paper, an ongoing activity to implement an inference scheme to derive in real-time surface shortwave radiative fluxes from GOES satellite observations is discussed. Presented is also an extensive evaluation of the product using ground observations from a variety of sources and examples how this product is used in a wide range of applications. The operational implementation of the methodology is facilitated by the ability to use selected operational outputs from the Eta model [Chen et al., 1996] as inputs to the inference scheme. The methodology to derive radiative fluxes from the GOES satellites has been implemented by NOAA/NESDIS in real time starting January 1996, and became operational in 2000. Evaluation against ground truth for 1996 (excluding the winter months for locations frequented by snow) indicates that the average rms errors on hourly, daily, and monthly timescales were about 88, 24, and 12 Wm−2, respectively. The satellite estimates used were obtained from the real-time runs, without adjustments for satellite sensor degradation and without “tuning” to local conditions. The accuracy of this product has been found to be better than what is currently achievable from numerical models. Therefore the product in its present form has been found useful in a wide range of studies and applications. Still, many unresolved issues require further development such as improved information on snow cover and cloud detection over snow. Calibration of the satellite sensors is another source of errors. To obtain accurate assessments of satellite capabilities to estimate surface radiative fluxes, there is a need to improve the quality of the ground observing stations, as well as their representativeness for each satellite grid cell. Work is in progress to remove known problems, and to expand the validation process to other climatic regions of the United States.
 The differences found in the observations from the two adjacent radiometers at Bondville, Illinois were investigated in depth (J. Augustine, personal communication, 2002). One station belongs to the SURFRAD network, which operates an Eppley PSP, while the other is operated by the Illinois Water Survey Network, under the direction of S. Hollinger, using alternating black and white sectors as detectors. Initially, we compared observations for November and December 1995 with satellite inferred values and found that the SURFRAD measurements were always lower than those from the Bondville BEARS site (here after, the Bondville instrument). The initial suspicion was that the differences appeared to be the result of a calibration offset in one of the instruments, because the absolute difference between measurements increased with the degree of downwelling irradiance. The calibration of the SURFRAD instrument was traced to National Renewable Energy Laboratory (NREL) during the summer of 1995. To resolve this issue, it was agreed (J. Augustine and S. Hollinger), that after the routine SURFRAD instrument exchange at Bondville, all instruments will be sent to Boulder, Colorado to be compared to the Surface Radiation Research Branch (SRRB) of the NOAA's Air Resources Laboratory standard set of Eppley PSP instruments. For this purpose, the SRRB standard PSPs were recalibrated at NREL during September of 1996. The new sensitivity factors were applied to the standards, and the sensitivity factors that have been used while they were at Bondville, were applied to the returning pyranometers. The intercomparison has shown that qualitatively, all instruments saw the same signal, and none has shown anomalous behavior. It is usual for these instruments to produce slight negative signals at night owing to the inner dome cooling to space. The magnitude of these signals was evaluated, and found to be in a range that did not explain the differences between the two radiometers. Subsequently, the returning radiometers were gauged against each other, by forming ratios to the mean of the three standards. During two clear days of calibration, ratios of the SURFRAD radiometers to the mean of the standards are clustered near 1.0, but they did show a slight cosine error with respect to that of the standards. The Bondville instrument showed a systematic offset and a more severe cosine problem. During the cloudy period of the calibration, the SURFRAD instruments and the Bondville instrument showed unexplained random differences with respect to the standards. The mean ratio at 50° of solar zenith angle (at this angle NREL computes sensitivity factors) for one SURFRAD instrument was 0.994, while for the other it was 0.999. For the Bondville instrument, it was 1.051, which translates to a bias of 5.1%, close to what we have.
 The work at the University of Maryland was supported by grants NA86GPO202 and NA06GPO404 from NOAA/OGP and by NOAA grant NA57WC0340 to University of Maryland's Cooperative Institute for Climate Studies (R. T. Pinker, PI). The work on this project by NCEP/EMC, NWS/OHD, and NESDIS/ORA was supported by the NOAA OGP grant for the NOAA Core Project for GCIP/GAPP (co-PIs K. Mitchell, J. Schaake, D. Tarpley). The work by NASA/GSFC/HSB was supported by NASA's Terrestrial Hydrology Program (P. Houser, PI). The work by Rutgers University was supported by NOAA OGP GAPP grant GC99-443b (A. Robock, PI). The work by Princeton was supported by NOAA OGP GAPP grant NA86GP0258 (E. Wood, PI). The work by NCEP/CPC was supported by NOAA/NASA GAPP Project 8R1DA114 (W. Higgins, PI). We thank NOAA for the SURFRAD data that were provided to the project at no cost. Thanks are due to Q. -H. Li for assistance in the evaluation of the satellite retrievals; to Dr. S. Hollinger for arranging the prompt transfer of ground observations from the Illinois Water Survey; Drs. J. DeLuisi, J. Augustine, and C. Long for helpful discussions of the SURFRAD data; Dr. W. D. Sellers, for providing the University of Arizona, Tucson data; to Dr. K. Kunkel, for providing results from as yet unpublished work; and to Kevin Fitch, Navid Golpayegani and Wen Meng for building and maintaining the GCIP/GAPP Web site. The authors thank Byron Raines (Raytheon, STX) for his sustained effort in building and maintaining the NESDIS software system. Some figures were drawn with GrADS, created by Brian Doty.