Extraterrestrial solar spectrum 360–1050 nm from Rotating Shadowband Spectroradiometer measurements at the Southern Great Plains (ARM) site



[1] Two years of Langley extrapolations made from the Southern Great Plains (SGP) site of the Atmospheric Radiation Measurement (ARM) Program using two very different RSS instruments and a NIST-derived irradiance scale show larger extraterrestrial solar irradiances in the 400 to 600 nm domain by as much as 4.5% compared to the Neckel and Labs [1984] data. Our results are more congruent with that of Thuillier et al. [1998a, 1998b] in this domain but do not show larger irradiances than the Neckel and Labs data at wavelengths less than 400 nm. In addition to the question of the true extraterrestrial solar spectrum, the data we present here are directly useful for the interpretation of spectral measurements calibrated with current National Institute of Standards and Technology (NIST) irradiance standards, for which we believe this represents the most extensive determination of the solar irradiance.

1. Introduction

[2] The most commonly used Solar spectral data for optical wavelengths have been from the measurements of Neckel and Labs [1984]; the WMO consensus spectrum is based on these data in the wavelength range 330 to 869 nm [Wherrli, 1985]. These irradiance data were a by-product of a measurement campaign to study solar limb-darkening; the transfer of the absolute irradiance scale depended on calibrations of ribbon-filament radiance lamps. More recent measurements include those of Lockwood et al. [1992], Burlov-Vasiljev et al. [1995], Thuillier et al. [1997, 1998a, 1998b], and ours [Harrison et al., 1999]. These measurements disagree in systematic ways by significantly more than each measurement's reproducibility. The problem is a general one in spectroradiometry; systematic errors associated with both the detection system and calibration pathway dominate the error budget and are often difficult to understand or eliminate.

[3] The Rotating Shadowband Spectroradiometer (RSS) is a field instrument developed for the ARM program, which employs a Charge-Coupled Device (CCD) spectrograph spanning the wavelength range 360 to 1100 nm with a shadowbanded fore-optic. The RSS provides spectrally-resolved direct-normal, diffuse-horizontal, and total-horizontal irradiances. These three components are guaranteed to have the same passbands and responsivities; thus Langley calibrations can be used for the horizontal surface irradiances as well, producing self-consistent data for radiative transfer calculations.

[4] RSS measurements at SGP have been taken with two versions of the instrument. In 1997 and 1998 the “first generation” RSS instrument was deployed, with a 512 pixel linear NMOS (N-channel Metallic Oxide Semiconductor) array. Most RSS data used to date for a variety of remote sensing and closure study efforts are from this first-generation RSS: Michalsky et al. [1999], Min and Harrison [1999], Min et al. [2001], Mlawer et al. [2000], and Schmid et al. [1999]. In the summer of 1999 we deployed a substantially revised instrument. This “second-generation” instrument uses a higher-performance 1024 × 256 CCD array; with this better detector a variety of other instrument design compromises could be revisited and performance substantially improved in resolution, out-of-band-rejection, and dynamic range. Other changes made on the basis of experience improved wavelength stability. Harrison et al. [1999] describe the RSS, compare data from the two instrument generations (which were operated side-by-side for a transition period at SGP), and show the first of the two Langley extrapolation data sets discussed in this paper.

[5] Here we show a more extensive Langley extrapolation data set taken with the second-generation instrument, together with more extensive calibration, yielding results closely congruent with our earlier effort [Harrison et al., 1999].

2. Langley Regression and Estimation of the Extraterrestrial Instrument Response, V0

[6] SGP is a low-altitude midcontinental site, with larger aerosol optical depths and associated variability than high-altitude sites commonly preferred for Langley extrapolation efforts. The ARM science mission for the RSS is continuous operation at the SGP facility; consequently calibrations of the extraterrestrial instrument response needed for sun-photometry are derived from the routine data using an objective Langley algorithm developed by Harrison and Michalsky [1994], and tested in a Sun photometer intercomparison reported by Schmid et al. [1999]. On-site calibrations derived from the available Langley events are practical only because the continuous operation yields a sufficient number so that statistics partially compensate for the higher event-to-event variance.

[7] Here we analyze the “Langley events” observed by this second-generation RSS at SGP from 27 July 1999 through 23 May 2000. This period spanned three irradiance calibrations of the instrument and contained the fall of 1999, which was unusually cloud-free. In routine operation at SGP the RSS makes observations at 60 s intervals. The Langley regressions were done for two air mass ranges: 2 to 6 and 2 to 4.5, to aid statistical tests of aerosol-column impacts on the inferred extraterrestrial irradiances.

[8] The 2–6 air mass range was used by Harrison et al. [1999] and will be shown first to permit closest comparison of the new observations with the earlier ones. Using its standard screening criteria the objective Langley algorithm returned 122 “Langley events” for the 2–6 air mass range analysis, and 115 for the 2–4.5 air mass range; of these 90 events match. The objective algorithm accepts events on the basis of the fraction and residual variance of observations within the air mass range that survive its tests to eliminate cloud passages. The majority of the events do have internal cloud-passage periods that the algorithm rejects; it is not surprising that some events survive when tested over one air mass range, but not the other. The data reported by Harrison et al. [1999] were 27 events.

[9] Each Langley regression is done on the raw instrument output data (instrument counts for each pixel), and yields an optical depth τ, an uncalibrated extraterrestrial instrument response V0, and associated regression statistics for each pixel (wavelength). All the V0 data as retrieved are immediately normalized to 1 astronomical-unit (AU) distance, by multiplying by R2, where R is the Earth-Sun distance in AU. To avoid confusion on this issue following references to normalized extraterrestrial response are to Vn = V0 * R2. A Langley optical depth spectrum taken by the second-generation RSS from the morning of 2 November 1999 at the SGP site is shown in Figure 1. The optical depths are the slope of the Langley regression and do not depend on any instrument calibration. (The wavelength calibration of the instrument is done separately against low-pressure line-emission sources, and not discussed here. It controls the abscissa of the figure, but does not affect the Langley regression.) The top panel of Figure 1 is a log-log plot showing the large range of optical depths observed, and the successive subtraction of the O3 and NO2 contributions inferred from the measurement. The 300 DU of O3 inferred for this event is common, and negligibly different from the TOMS estimate of 297 DU for this location and time. However, the 0.55 DU of NO2 inferred for this event (from first-difference correlation of the domains 400 − 440 nm of both the τ spectrum and the NO2 cross-sections, i.e., conventional differential-optical absorption spectroscopy, DOAS) is an uncommonly large column. For the purposes of the Langley extrapolation of V0 the partition of the extinction of these “weak absorptions” does not matter so long as their columns are invariant during the course of the time series used for the Langley regression. Most of the remaining structure visible in the top panel is dominated by strong absorption bands of H2O and O2. Within these bands curve-of-growth causes Langley regression to fail; both τ and V0, are underestimated. The underdrawn dotted line is the aerosol extinction inferred as an Angstrom distribution. This case is representative of the structure and signal-to-noise seen in the Langley regressions done with RSS data, but unusual for the low aerosol optical depth (0.011 at 500 nm) and high column NO2 inferred.

Figure 1.

An optical depth (τ) spectrum from a Langley regression; data at SGP 2 November 1999.

[10] The lower panel of Figure 1 shows the short-wavelength end of the optical depth spectrum in greater detail. Weak absorption features of O2-O2 are seen [Pfeilsticker et al., 1997; Michalsky et al., 1999] and the correlation of the optical depths with the superimposed NO2 cross-sections is apparent to the eye in the range 400 to 440 nm, where other spectral features do not dominate. This high-NO2 case was chosen in part so that these are perceptible, but the optical depth due to NO2 remains very small. We show these here in part as a demonstration of instrument performance; the differential sensitivity (and also wavelength stability through the period) are sufficiently good to measure NO2 via DOAS in this wavelength range at the relatively low air masses observed in these Langley events. More commonly column NO2 is measured using low-solar-elevation observations to produce effective air mass amplifications near 25.

[11] The K and H solar lines appear ubiquitously as negative residuals in these optical depth spectra, with nearly uniform magnitudes. In the simplest consideration they should not. We attribute these as consequences of both the “Ring Effect” (weak inelastic scattering by atmospheric gases) and less-than-perfect out-of-band rejection in the spectrometer. The inelastic scattering increases as the air mass increases, producing an apparent negative contribution to the local optical depth. Increasing air mass also attenuates the shorter wavelengths far more than longer ones, and very weak stray light contributions from longer wavelengths could manifest themselves similarly in these Fraunhofer lines. We believe that the Ring-Effect contribution is larger than the stray-light contribution for this second-generation RSS data, on the basis of both an estimate from our measured instrument slit-scattering functions (data from Harrison et al. [1999]) and a comparison to data from the earlier instrument. For this reason the region below 400 nm is excluded from our NO2 retrieval, together with the weak O2-O2 bands. (Note that these Ring-Effect perturbations on the optical depth are larger than the differential cross-sections of the NO2, for this case where the NO2 is ≈ 3 times larger than the expected stratospheric background; again, NO2 absorptions are small.) With the NO2 removed the residual variance of the remaining optical depth is less than ≈ 0.002 RMS; some of the remaining variability is due to inexact matching of NO2 cross-section data to our instrument's observing function, and other weak trace-gas absorptions not considered.

[12] A second reason the NO2 impacts on the optical depth spectrum are shown here is as part of a discussion of the potential error in the extrapolated V0 due to changing NO2 column during the Langley interval. This error is negligible compared to other errors in our end results, but we defer this until other results are shown.

[13] The Langley-extrapolated Vn data from the 122 air mass-range 2–6 events are shown top-to-bottom in a false-color image in the middle panel of Figure 2. These are the sequential events, without the varying intervals between them. There are 23 pairs of events (≈ a third of the total events), which are the morning and afternoon of the same day. The longest interval between events is 12 days. Pixels are nonlinearly short-wavelength to long, left to right. Each horizontal bar of pixels is a color-coded single Vn spectrum with sequence black, brown, red, gray, violet, white. The thresholds were adjusted to maximize the perception of gradients. A raw Vn spectrum (average of three) is plotted conventionally for the comparison in the top graph with the same pixel abscissa registration as the false-color presentation below it. The bottom graph shows a calibrated ET spectrum versus wavelength for visual comparison; note that the wavelength mapping from pixels is nonlinear (compare features to top panel).

Figure 2.

Vn spectra from Langley regression 2–6 air masses; second-generation RSS at SGP.

[14] The undeviating vertical lines of the absorption features in the middle panel emphasize the long-term wavelength stability. No wavelength fitting has been necessary with these data. Horizontal “bars” are outlier Langley events, shown further below. Black spots are data where the regressions for individual pixels exceeded a variance criterion; these are not accumulated in further averaging. The lack of trend top to bottom is a general indicator of instrument throughput stability; the apparent exception seen around pixel 820–847 lies in the 820 nm H2O band. Within strong-absorption bands curve-of-growth causes Langley regression to fail; both τ and V0 are underestimated. The seasonal trend in H2O from July to December is then evident in these domains in the Vn(λ) shown in Figure 2.

[15] On closer examination instrument trends in Vn with time can be discerned in this rich Langley regression series. We compute the trends within the Langley series by applying linear and second-order least squares polynomial fits. The ratio of the fitted Vn is then computed for two date-pairs: 7 July 1999 and 10 March 2000, and then 8 February 2000 and 10 March 2000. During this process (and for the subsequent computation of mean E0(λ)) outliers are trimmed from the Langley pool by keeping only the 85% of the events closest to the median. On these three dates field irradiance calibrations were done on the instrument against our portable LiCor calibrator, which in turn had been calibrated against a pool of NIST FEL lamps [Kiedron et al., 1999] and also as part of the ARESE II field calibration efforts (P. Kiedron, private communication, 2002). These data show that the Langley estimates of the trend (a trend in instrument responsivity) closely match the ratios of responsivities seen against the portable calibrator, as shown in Figure 3. The close congruence of the linear and second-order fit predictions demonstrates that a linear trend explains the observations well. This conclusion is supported by the nearly identical values for the estimated 1-σ fractional precision of the residuals from the regression for the two schemes shown in Figure 4; the second-order scheme does no better except in the H2O bands, where it fits the seasonal artifact of the H2O column discussed previously.

Figure 3.

Ratios of Vn (normalized to 1 AU), 2–6 air mass range; second-generation RSS at SGP.

Figure 4.

Precision of detrended Vn estimates from Figure 3.

[16] Figure 4 shows a direct measure of the intra-Langley variance within the remaining 85% of the Langley pool. The high-resolution structure seen at wavelengths below 400 nm is associated with Fraunhofer structure and diffuse-sky scattering variability for the Ring-Effect. The cluster of small peaks near 430 nm is also correlated with Fraunhofer structure rather than the strongest differential cross-sections of NO2 (which have maxima continuing to nearly 450 nm).

[17] Figure 5 shows our extrapolated solar irradiances from the 1997 effort (Harrison et al. [1999]; identified as “RSS-512,” in blue) and the results of our 1999–2000 observations from the new instrument (“RSS-1024,” in red) both presented as ratios to the “Old Kurucz” extraterrestrial spectrum from MODTRAN 3.4. This extraterrestrial spectrum has been superseded, but is used here for a common denominator for both as it was used for our earlier work, and because we understand it to be observations from the McMath Solar Observatory (very high resolution) envelope-fitted to the Neckel and Labs [1984] data in our domain of interest. (Outside the H2O bands there is indeed very close agreement between these two.) We use this spectrum for the comparison (rather than directly against the WMO consensus or the Neckel and Labs [1984] data) because its high spectral resolution allows us to apply our instrument observing functions so that it is brought close in resolution to what our instrument would observe. Note that at very short wavelengths the rich solar absorption spectrum, interacting with small errors in wavelength registration or instrument observing function, cause these ratios to appear noisy. This is a common difficulty when two spectra taken by different instrument are ratioed, unless the measurements are both substantially degraded in resolution. The use of the “OldKur” data allow us to present these ratios as a surrogate for Neckel and Labs [1984] without further degrading the resolution of our instrument.

Figure 5.

1997 and 1999–2000 extrapolated solar irradiances/“Old Kurucz.”

[18] The fidelity with which we reproduce the inferred ET spectrum in the two experiments is very encouraging. These two experiments are completely independent in the instrument used (the second-generation RSS having 3 times the resolution, completely different detector and electronics (and hence residual nonlinearities), responsivity, out-of-band rejection, dynamic range, angular fore-optic calibrations), and the pool of observations. They share only the calibration to the ARM-NIST irradiance lamp pool, but that done two years apart. In the wavelength range 390 to 800 nm the largest apparent discrepancies that are not obvious curve-of-growth features are in the domain of the gamma band of 02 (near 630 nm) and a weak H2O feature at slightly longer wavelengths. (These can be seen in the top panel of Figure 1.) Discounting these, the smoothed discrepancy between our two sets of observations is congruent with the 0.1%–0.2% (increasing to shorter wavelengths) 1-σ estimate of the precision of the Vn values from the significantly smaller 1997 data set.

3. Impacts of NO2 Variability on the Inferred E0(λ)

[19] The general envelope seen in Figure 5 appears roughly similar to the NO2 cross-section. However, multiple lines of evidence demonstrate that it is not caused by NO2 variability within the Langley events (≈1 hour). The detailed structure of the features in Figure 5 within the strong domain of the NO2 cross-sections do not match for the two experiments, nor exhibit significant differential correlations with the NO2 cross-section. If all the Langley events had a NO2 column of 0.5 DU (a near extremal value seen in our data) at 6 air masses, with a linear destruction to zero at 2 air masses during the Langley event, the impact on the E0 would be ≈0.5%. It is implausible that NO2 variability would be so systematic (particularly for the larger NO2 columns seen, which must be dominated by tropospheric columns, and hence with strong advection variabilities), and particularly so across the morning and evening Langley events within the pool, or across the two experiments two years apart. In the absence of implausibly consistent trend of NO2 versus air mass during Langley events, Figure 4 shows that the upper bound for NO2-induced error on the E0(λ) is 0.1%. This is much less than other sources of error.

4. Estimation of Systematic Error Due to Aerosol Optical Depth Interferences

[20] The reproducibility demonstrated here is so good that the true error is almost certainly dominated by systematic issues not addressed simply by looking at the variances between the two independent efforts. Here we discuss potential systematic errors arising from the Langley extrapolations, and in a following section we further discuss potential systematic errors of our irradiance scale and transfer to the instrument.

[21] Schmid et al. [1999] report an intercomparison of sunphotometers at the SGP in 1997, which included the first-generation RSS. While several of the participating sunphotometers used Vn calibrations obtained from high-altitude measurements conducted elsewhere, the RSS estimations of the aerosol optical depths were based solely on the Vn results of the Langley pool discussed by Harrison et al. [1999] and repeated here in Figure 5. The congruence of the aerosol optical depths obtained are thus an effective test for systematic discrepancies between either data quality or reduction methods among these very different instruments and sources of calibration. The first-generation RSS data produced aerosol optical depths that agreed with the AATS-6 instrument (taken to have the best calibration) to within the 1-σ error limit predicted from the variance of the Langley pool available for that intercomparison. This gives us confidence that we do not have a basic error in application of the Langley process.

[22] Figure 6 shows a scattergram of the Vn versus total optical depth at 500 nm for the 2–6 air mass range cases, the data from Figure 2. The abscissa value at the left limit of the plot is set to 0.1436, the Rayleigh optical depth for 500 nm from Kasten's revised approximation [Kasten and Young, 1989]. The aerosol optical depths are thus approximately the total optical depth minus a further small decrement for Chappuis-band O3; with the assumption of a climatological O3 column of 339 DU the mean aerosol optical depth of the observations is 0.076 at 500 nm. The low correlation to the linear-fit is evident; the correlation coefficient is −0.0148. Thus there is no direct evidence for any systematic effect due to aerosol optical depth.

Figure 6.

Scattergram of Vn versus τ at 500 nm.

[23] Nonetheless we were not completely relieved, because a modeling effort shows that aerosol optical depths can affect the extrapolated Vn values when the aerosol optical depth is significant, and the aerosol is not distributed uniformly in altitude. The latter is of course almost a certainty. In this case if the air mass factor appropriate for a uniformly mixed species (e.g., Kasten's approximation) is assumed for simplicity as we have done for these data then the V0 (and hence directly Vn) estimates exhibit a systematic error which grows approximately as m2, where m is the limiting air mass factor used in the regression. In order to provide a direct assessment of the potential magnitude of such systematic effects on our data we re-ran the Langley regressions over the air mass range 2–4.5 and compared the ratios of Vn. This is shown in Figure 7. We delay some further discussion to the conclusions.

Figure 7.

Ratios of V0 determined from two air mass ranges.

Figure 8.

ARM's pool of NIST 1000 Watt FEL lamps.

5. ARM/NIST Irradiance Scale

[24] We maintain the ARM spectral irradiance scale, and transfer to the SGP site irradiance calibrator(s). The issue of ARM spectral irradiance scale is now particularly important as all ARESE II spectral calibrations have been done against this “ARM/NIST working standard.” For the '97 IOP we established this as the mean of the three NIST FEL lamps F403–F405, discarding F340 as an evident outlier. Measured spectral ratios of these lamps' outputs relative to their stated output are shown in Figure 8, from Kiedron et al. [1999]. The red and blue curves show the repeatability of these measurements 1 day apart. Repeatability over 6 months is not significantly worse.

[25] NIST recalibrated our lamp F340 in July 1999. This recalibration demonstrates that the mean of the three ARM working lamps F403 through F405 is indeed very close to NIST's working irradiance source, and allows us to use this pool as representing the NIST standard despite the distressing variance among the individual lamps with their production calibrations by NIST seen here.

6. Discussion and Conclusion

[26] The revised second-generation RSS deployed to the SGP site of the ARM program is a significantly improved instrument over the first in all regards except long term stability of the instrument's responsivity. We did not observe a trend in the responsivity of the first-generation instrument, although our detection limits are somewhat better now than then. (In this regard the interval of the 1997 Langley series and the Intensive Operation Period where the Sun photometer intercomparison took place was much shorter than the 1999–2000 data series analyzed here for the newer instrument.) However we are confident that had the first-generation instrument exhibited changes in responsivity comparable to those seen here we would have detected it. Hence understanding the causes and preventing such drifts remains a high-priority technical effort for us with regard to the RSS, which we have not fully resolved. We believe that the likely cause was fundamental change of responsivity of the CCD-array detector itself (which is different in this newer instrument than the old one, see Harrison et al. [1999] for more technical detail), but are not yet certain whether it was caused by a subtle defect in the particular part used (which was an engineering sample in this early effort), our handling/protection of it, oxidation or H2O damage (the instrument was operated with a continuous dry air purge), or is intrinsic to the open-polysilicon electrode device. We have efforts underway to diagnose and ameliorate this, but the testing is necessarily slow.

[27] Nonetheless the responsivity trends in the second-generation instrument were slow, linear over the operational epoch, and confirmed by both Langley trends and the series of irradiance lamp calibrations. The large Langley pool obtained in the fall of 1999 reproduce the features of the Extraterrestrial spectrum we observed in 1997 to better than 0.25% at most wavelengths. Taken in total, we think that these observations are remarkably consistent and reproducible, and that this achievement with two independent campaigns, each with physically very different instruments in terms of the potential residual systematic artifacts, gives us confidence that the results are robust as demonstrated in Figure 5.

[28] These data show the sun to be systematically brighter in the wavelength range 400 to 600 nm than the Neckel and Labs [1984] results, which are the basis for both the WMO consensus [Wherrli, 1985] and to which the “old-Kurucz” data were envelope-fitted. Thuiller et al. [1998a, 1998b] show data qualitatively more similar to ours than to Neckel and Labs, but not agreeing in close detail. In particular the data from Thuiller et al. are brighter at wavelengths shorter than 400 nm than ours, the data of Neckel and Labs [1984], and the results of SUSIM and ground-based measurements reported by Grobner and Kerr [2001]. However, systematic uncertainties of calibration are larger at the shorter wavelengths for both our effort and Thuiller et al., and in our case aerosol impacts on Langley extrapolation are moderately larger. Given these many issues, we think the discrepancies below 400 nm remain uncertain at this time.

[29] The recalibration of our lamp F340 by NIST establishes that the ARM/NIST lamp pool consensus is within 1% of NIST's internally held standards. Our repeated transfers to the field instrument are congruent with Langley-observed trends and show a variability below 1%. While the SGP site is at low altitude and is relatively polluted, and so certainly not the ideal place to conduct a Langley effort to measure the extraterrestrial spectrum, the aerosol interferences can plausibly explain no more than approximately one-third of the discrepancy. Given the independent Thuillier et al. [1997, 1998a, 1998b] observations and our demonstration of reproducibility of results versus NIST irradiances from data two years apart we believe that an upward revision of the solar spectrum is now warranted in the 400 to 600 nm domain. Further efforts are clearly needed to improve accuracies, particularly at shorter wavelengths. The results presented here are the best current estimates of a solar irradiance function congruent with the NIST spectral irradiance scale, and these values are necessary for the interpretation of many observations within the ARM program and elsewhere tied to NIST calibrations.

[30] More recent versions of MODTRAN contain an Extraterrestrial Spectrum identified as “newkur” which is much closer to a pure modeled result for the solar emission than “oldkur” [Fontenla et al., 1999]. Figure 9 shows our results versus “newkur” in a more limited spectral domain. The ARM program has adopted “newkur” as its default ET spectrum for modeling purposes. It is apparent however that the discrepancy near 430 nm (the g-band of calcium) in “newkur” is simply not plausible. None of the instrument observations see this, and the structure seen in this ratio is unlikely as an individual instrumental error, let alone a common systematic error in all the measurements. Similar comments apply (though less forcefully) at shorter wavelengths for the H & K bands of calcium, and to the envelope shape. Modelers prefer this spectrum for its high resolution and pedigree, but it is clearly not able to provide good closure for band-model comparisons against terrestrial observations of atmospheric transmission unless this Solar spectrum or the observations are “corrected” to force congruency [Mlawer et al., 2000].

Figure 9.

1997 and 1999–2000 extrapolated solar irradiances, ratios/“New Kurucz.”