A detailed spectrally resolved extraterrestrial solar spectrum (ESS) is important for line-by-line radiative transfer modeling in the near-IR. Very few observationally based high-resolution ESS are available in this spectral region. Consequently, the theoretically calculated ESS by Kurucz has been widely adopted. We present the CAVIAR (Continuum Absorption at Visible and Infrared Wavelengths and its Atmospheric Relevance) ESS, which is derived using the Langley technique applied to calibrated observations using a ground-based high-resolution Fourier transform spectrometer (FTS) in atmospheric windows from 2000 to 10,000 cm–1 (1–5 µm). There is good agreement between the strengths and positions of solar lines between the CAVIAR and the satellite-based Atmospheric Chemistry Experiment-FTS ESS, in the spectral region where they overlap, and good agreement with other ground-based FTS measurements in two near-IR windows. However, there are significant differences in the structure between the CAVIAR ESS and spectra from semiempirical models. In addition, we found a difference of up to 8% in the absolute (and hence the wavelength-integrated) irradiance between the CAVIAR ESS and that of Thuillier et al., which was based on measurements from the Atmospheric Laboratory for Applications and Science satellite and other sources. In many spectral regions, this difference is significant, because the coverage factor k = 2 (or 95% confidence limit) uncertainties in the two sets of observations do not overlap. Because the total solar irradiance is relatively well constrained, if the CAVIAR ESS is correct, then this would indicate an integrated “loss” of solar irradiance of about 30 W m–2 in the near-IR that would have to be compensated by an increase at other wavelengths.
 Knowledge of the extraterrestrial solar irradiance is important for radiative transfer modeling and satellite applications. The absorption of solar radiation is an important component of the Earth's energy budget [see, for example, Trenberth et al., 2009], and detailed calculations require a sufficiently accurate spectrally resolved extraterrestrial solar spectrum (ESS) [Chance and Kurucz, 2010]. In this paper, we are concerned with the extraterrestrial solar irradiance in the near-IR, from 2000 to 10,000 cm–1 (1–5 µm), which is a region of significant atmospheric absorption especially by water vapor.
 Over the years, the ESS in different spectral regions and at different resolutions has been measured from the ground [e.g., Burlov-Vasijev et al., 1995], using aircraft [e.g., Arvesen et al., 1969], using balloons [e.g., Hall and Anderson, 1991], and spacecraft [e.g., Thuillier et al., 2003]. More recently, Thuillier et al.  built a composite solar spectrum (henceforth referred to as “Thuillier” ESS) from 0.1 to 2400 nm with a spectral resolution of 0.5 nm in the near-IR. The uncertainty of this spectrum is quoted as 3% (coverage factor k = 2, 95% confidence interval) on the average in the visible and near-IR (4200–15,000 cm–1) [Thuillier et al., 2003]. Although this spectrum has become an accepted reference in the spectral range 400–2400 nm (about 4200–25000 cm–1) [see Harder et al., 2010; Chance and Kurucz, 2010], its spectral resolution is lower than the spectral resolution used for line-by-line radiative transfer calculations. However, it will be used here for its absolute level.
 The ESS has also been obtained by calculations using models of the Sun's atmosphere. For example, Kurucz  produced an ESS using a model of the Sun's photosphere (referred to as “Kurucz-modeled” ESS hereafter) in the spectral region 0–50,000 cm–1 with an output step of 0.1 cm–1. Although the spectral resolution of the Kurucz-modeled ESS is relatively high, it does not represent the solar spectrum in detail because only the Sun's photosphere is represented. In fact, the strength, wavelength, and broadening of every single line in the Kurucz-modeled ESS was adjusted by hand (R.L. Kurucz, personal communication, 2010). Semiempirical models of the Sun's atmosphere have also been used to derive the ESS; for example, the ESS derived by Fontenla et al.  using the Solar Irradiance Physical Modeling (SRPM) system (henceforth referred to as “Fontenla” ESS) for different days of solar cycle 23 in the spectral range from 0.12 nm to 100 µm (about 100 cm–1 to 8.3 × 107 cm–1). The ESS produced by Fontenla et al.  is described as an “extremely high-resolution solar spectral irradiance”; the SRPM data presented here uses an output step of 0.1 cm–1, which is sufficient to identify the lines in the spectrum, although the detailed shape of these lines (which is not considered here) will not be so accurately represented because the highest-resolution output from SRPM has a spectral resolution of order 10–4 cm in the near-IR.
 Very few high-spectral resolution measurements of the ESS in the near-IR have been reported. An important high-spectral resolution (0.02 cm–1) measurement of ESS is that reported by Hase et al.  using the satellite-based Atmospheric Chemistry Experiment (ACE) Fourier transform spectrometer (FTS) (hereafter referred to as “ACE-FTS” ESS), which has a higher signal-to-noise than earlier similar measurements reported by Farmer and Norton . Because the ACE spectrometer was not calibrated and covered a relatively small spectral region (700–4430 cm–1), it cannot be used directly for quantitative atmospheric modeling studies. Instead, this spectrum is presented as a spectrum relative to some baseline. This is reported as “transmission”, although it is not a physical transmission, but rather a mathematical abstraction to separate the high-resolution FTS measurements from a smooth continuum background solar emission (F. Hase, personal communication, 2011). Kurucz  has produced an ESS (hereafter referred to as “Kurucz-observed” ESS) using FTS measurements taken at the Kitt Peak National Observatory, Tuscan, Arizona in two near-IR windows: 1560–1720 nm (about 5813–6410 cm–1) and 1920–2100 nm (about 4761–5208 cm–1). The spectral resolution of these measurements is 0.01 cm–1 and the uncertainty is quoted as 1–3%.
 Using ground-based observations for deriving the ESS has a number of advantages, such as relative ease of calibration, less limitation as to the power required for the instruments, and fewer constraints on the size and weight of instruments [Thuillier et al., 2004]. Ground-based measurements from high-altitude sites present an additional advantage of lower aerosol and water vapor optical depth, and provide more spectral intervals where the solar radiation can be measured. However, there are also some clear drawbacks (such as absorption by water vapor in the near-infrared region in particular) associated with using ground-based measurements to derive the ESS.
 Due to its high spectral resolution and broad spectral region covered, the Kurucz-modeled ESS has frequently been used in calculations of atmospheric absorption. However, it is important that the absolute level and structure of this solar spectrum be compared with measurements, which is the major aim of this work. Such a comparison has been done partially by Kurucz  in the two narrow near-IR windows stated above.
 In this work, high-resolution (0.03 cm–1) ground-based measurements of spectral direct solar irradiance using a Sun-pointing FTS under clear-sky conditions have been used to derive the ESS over broader near-IR windows between 2000 and 10,000 cm–1 (1–5 µm) than those covered by the Kurucz-observed ESS. These observations were made by the National Physical Laboratory (NPL), as part of the UK multi-institute consortium “Continuum Absorption at Visible and Infrared Wavelengths and its Atmospheric Relevance” (CAVIAR). This work was necessitated by some particular artifacts reported earlier by Tallis et al. . Their Figure 4 shows some “dips” in observed optical depths. These dips were attributed to errors in the ESS, which in that case was the Kurucz-modeled ESS. We used the Langley method (see section 3.1), which exploits surface observations of direct spectral irradiance made at several solar zenith angles within a day to derive the ESS. To our knowledge, the Langley method has not been applied to high-resolution observations of the ESS in the near-IR. To motivate the work described here, Figure 1 shows our derived ESS (called the “CAVIAR” ESS here) in a 10 cm–1 spectral region. Also shown are the Kurucz-modeled ESS and ACE-FTS ESS. Figure 1 shows disagreement between the CAVIAR ESS line positions and strengths and those of the Kurucz-modeled ESS. By contrast, there is a much better agreement between the CAVIAR and ACE-FTS ESS. An 8% difference in the overall ESS between the CAVIAR and Kurucz-modeled ESS is seen in this spectral region. Encouraged by the excellent agreement between ACE-FTS ESS and CAVIAR ESS, we then derived the ESS in the spectral region in which the NPL FTS measurements and calibration were made (2000–10,000 cm–1). NPL measurements made at a low-altitude site (Camborne, UK) were used to derive the CAVIAR ESS. In section 2, we discuss the measurement techniques. Section 3 focuses on the analysis method, and results are presented in section 4. Section 5 presents the merging of the CAVIAR ESS with other high-resolution extraterrestrial solar spectra and normalization of our final solar spectrum to the Thuillier ESS. This merging is needed because we could not observe the spectrum in spectral intervals with strong water vapor absorption. Finally, conclusions will be presented in section 6.
2 Experimental Methods
2.1 Field Campaigns and Fourier Transform Spectrometer Specification
 We use measurements from the CAVIAR field campaign at the UK Met Office Observation station at Camborne, UK (altitude 88 m above sea level, longitude 5.33° west, latitude 50.22° north) in summer 2008. NPL used a Sun-pointing high-resolution Bruker IFS 125M FTS to measure spectral direct solar irradiance at the surface. As many measurements as possible were taken over the course of a single day at different solar zenith angles (SZA) and under clear sky conditions (as seen by the naked eye). More details about the field campaign are given by Tallis et al. .
 We used FTS data in the spectral region 2000–10,000 cm−1 (1–5 µm) at spectral resolution of 0.03 cm−1. The FTS was interfaced with a solar-tracker, which automatically tracked the solar disc. An indium antimonide (InSb) detector, a calcium fluoride (CaF2) beam splitter, a boxcar apodization, and no optical filtering were the other specifications for the FTS. With these specifications, it took about 2 min for the FTS to carry out a single measurement consisting of 8 interferogram scans. Some measurements were taken using less than 8 scans, especially when clear sky conditions were not sustained. More details about the specifications of this FTS are given by Casanova et al. , Tallis et al. , and Gardiner et al. . Appropriate observations from two days (22 August 2008 and 18 September 2008) were chosen because they represented the most consistently clear days.
 Observations during these two days were selected if they satisfied two initial criteria. First, that to the naked eye there was clear sky conditions along the line of sight, and second, the detector's direct current (DC) did not vary by more than about 10% during a scan. This second criterion ensured that observations were rejected where the measurements might have been affected by clouds invisible to the naked eye or some instrumental effect such as momentary loss of alignment of the solar tracking optics. Lastly, the DC signals for all observations across a measurement day were plotted as a function of time, and any observation found to deviate upon visual inspection from the expected trend was rejected (i.e., there should be an inverse relationship between DC signal and the SZA).
2.2 Calibration of the Fourier Transform Spectrometer
 The FTS was calibrated using NPL's laboratory-based Ultra-High Temperature Blackbody at about 3000 K. Though lower than the average temperature of the Sun's photosphere (about 5800 K), the brightness temperature of the Sun measured by the FTS at the Earth's surface is close to this value. Hence the intensity on the detector and the signal levels throughout the detection electronics are consistent between calibration and measurements. A mobile radiance source called Transfer Standard Absolute Radiance Source was used to ensure that the calibration was valid during field measurements. When the main sources of uncertainty in FTS measurements (calibration source, the solar source including the center-to-limb variability, the FTS instrument and the external optics) have been combined, the measurement (k = 1, 67% confidence interval) uncertainty of the solar FTS measurement from 2000 to 10,000 cm−1 is between 3.3% and 5.9%. For more details on the derivation of this uncertainty and the calibration procedure of the FTS, see Gardiner et al. . The wavenumbers reported here are vacuum wavenumbers. We have verified that the position of the atmospheric lines of nitrous oxide, which are straightforward to detect in our measurements, agree with the positions reported on the HITRAN database [Rothman et al., 2009] to within the resolution of the observations (0.03 cm–1).
3 Analysis Methods
3.1 The Langley Method for Deriving the CAVIAR Extraterrestrial Solar Spectrum
 The ESS was derived using the Langley method, which exploits ground-based measurements of direct solar irradiance made at several SZA within a stable day (generally clear-sky conditions) [e.g., Liou, 2002]. This method was developed in the early 1900s by S.P. Langley at the Smithsonian Institute, USA to derive the total solar irradiance using broadband measurements. Because of its robustness, the Langley method has been used quite frequently. For example, Shaw et al.  used it to measure the atmospheric optical depth using a radiometer; Harrison et al.  used it to derive the ESS from spectroradiometer measurements in 360–1050 nm (9500–27,000 cm–1) spectral region; and Carbone et al.  used it to estimate the aerosol optical thickness using a spectrophotometer. Wang et al.  applied it to derive column nitrogen dioxide using high spectral resolution measurements at visible wavelengths. The application here of the Langley method to high-spectral resolution near-IR observations, to derive the ESS is, to our knowledge, novel.
 This method is based on the Beer-Bouguer-Lambert law (in which the spherical nature of the atmospheric layers and the refraction by the atmosphere due to changes in densities are neglected) given as
where ν is the wavenumber, Fν is the direct spectral irradiance measured at the surface, Fν0 is the spectral irradiance at the top of the atmosphere, τν is the spectral total vertical optical depth of the atmosphere, and m is the atmospheric airmass relative to the vertical. The airmass in equation (1) is equal to secθ, where θ is the SZA; the error in this expression increases with θ, with relatively small errors (less than 0.1%) for SZA up to about 80° [Iqbal, 1983]. Because no observations were made at SZA ≥80° the error is negligible compared to other error sources described here.
 Taking natural logarithms of equation (1), gives
 The Langley method assumes that if the properties of the atmosphere (such as humidity, surface pressure, aerosols, and cirrus) remain constant during the measurement period, then ln Fν will vary linearly with m. Thus, a plot of ln Fν against m (called the Langley plot) would be a straight line. To obtain the spectral irradiance at the top of the atmosphere, a linear least-squares fit was performed between ln Fν and m through the wavenumbers of interest. Statistical information such as the root-mean-square error, and the standard deviations of the regression coefficients (intercept, ln Fν0 and slope, τv) were also obtained during the linear regression. This line was extrapolated to m=0, which then yields Fν0. An important feature of the Langley method (because it uses multiple measurements at different SZA) is that the derived Fν0 is independent of the optical depth (due to, for example, water vapor or undetected thin cirrus) provided that this remains constant during measurements. This would not be the case if Fν0 was derived using a single FTS measurement, by trying to remove the influence of the atmosphere using, for example, modeled optical depths. Note that the slope of the Langley plot gives the total atmospheric optical depth, τν, but this is not considered here.
 The Langley method was not attempted in many spectral intervals within water vapor band centers as insufficient radiation was incident on the detector. However, it is still desirable to derive the extraterrestrial spectral solar irradiance as close to the band centers as possible. A comparison was made between wavenumbers where any changes in water vapor amount are expected to affect the transmission of solar radiation (line centers) and regions where such should have little or no effect on the transmitted solar radiation (microwindows).
3.2 Sources of Error in the CAVIAR Extraterrestrial Solar Spectrum
 In this section, we discuss limitations in using the Langley method, which lead to uncertainty in the derived ESS. One limitation was the sparsity of the measurements because data from only two days were used as stated in section 2.1. Confidence would have been greater had more days been available. Other limitations are discussed below.
3.2.1 The Range of Solar Zenith Angles or Airmass
 There is an ideal range of airmasses on which to do a Langley analysis. Low airmasses correspond to low SZA, which changes slowly with time as compared to higher SZA, and hence will be more affected if there are any changes in the atmospheric conditions. On the other hand, higher SZA observations are also not ideal because the airmass becomes more uncertain due to atmospheric refraction. Harrison and Michalsky  recommend that the range of airmasses suitable for Langley analysis should be from 2 to 6. The range of airmasses used here are 1.3 to 3.4 on 22 August 2008 and 1.5 to 5.3 on 18 September 2008, which partially fulfills this criterion. There are more “low” airmasses (i.e., those less than 2) on 22 August 2008 observations than on 18 September 2008 so that they are more likely to be affected by changing atmospheric conditions. However, the filtering criterion discussed in section 3.3 helped in mitigating this effect.
3.2.2 The Fourier Transform Spectrometer Scans
 As mentioned in section 2.1, a single FTS measurement was obtained from an average of a number of interferogram scans. For Langley analyses, it is desirable that the time taken to complete these scans should not be too long to ensure that SZA does not change significantly during an individual measurement. Harrison and Michalsky  recommend that the total scan time should not exceed 5 min. Within this time frame, it is clear that the higher the number of interferogram scans, the better the measurements for a given spectral resolution. Not all the observations at the same spectral resolution from our field campaigns were recorded with the same number of scans. The measurements of 22 August 2008 were averaged over 2 scans, while measurements of 18 September 2008 were averaged over 8 scans; this means that the Langley analyses for 18 September 2008 are more reliable than those of 22 August 2008.
3.2.3 Fourier Transform Spectrometer Measurements and Calibration
 The solar FTS measurement and calibration and their uncertainty were briefly discussed in section 2. This uncertainty applies equally to the Langley analyses, so that the solar FTS measurement (k = 1, 67% confidence interval) uncertainty in the CAVIAR ESS is also from about 3.3% to 5.9%.
3.2.4 Statistical Errors
 The presence of varying clouds and aerosols during the measurement periods will affect the Langley analyses. Clouds, especially cirrus, were definitely present during our measurements particularly on 22 August 2008. Efforts have been made to eliminate contamination as discussed in section 2.1, but it is unlikely that all “contaminated” observations were filtered out. This is another reason for having more confidence in our observations of 18 September 2008 than that of 22 August 2008. The variation in the cloud and aerosol amounts and the variation of the measured irradiance during the course of the FTS scans will lead to errors in the derived ESS. The errors introduced by these factors were quantified by studying the statistical errors from the Langley analyses.
 The standard deviations of ln Fν0 at m=0 have been used to put a statistical error on the derived ESS. Using the observations from 22 August 2008, the errors (95% confidence interval) introduced in the derived ESS by any deviation of the y-intercept is about 2.0%. For observations from 18 September 2008, the errors (95% confidence interval) are about 0.5%. These statistical errors are not spectrally dependent.
3.2.5 The Effect of Systematic Changing Atmospheric Optical Depth
 Any systematic variation in the total atmospheric optical depth during the period of measurements will lead to a variation in the measured irradiance and consequently in the derived ESS. This systematic variation cannot be removed by linear regression and thus must be accounted for in the derived ESS.
 The atmospheric optical depth is due to a combination of different effects given as (ν used to indicate the spectral dependence has been omitted in this section for the sake of brevity)
where τR is the Rayleigh scattering optical depth, τg is the optical depth due to gaseous absorption, and τa is the aerosol optical depth. For clear-sky conditions, τR is negligible in the near-IR. Thus, only changes in τg and τa may affect the derived ESS.
18.104.22.168 The Effect of Changing Atmospheric Gaseous Amount
 All gases except water vapor were assumed to be constant. Any changes in the water vapor amount during the period of measurements will affect the Langley analyses at wavenumbers near the centers of the spectral lines. GPS data of the water vapor column from Camborne show that it changes systematically by about 17% on the 22 August 2008 and 7% on 18 September 2008 during the period of measurements used here. This again indicates that the 18 September analysis was more reliable.
 To investigate the effect of systematic changes in water vapor amount during measurements on the derived ESS, we examine the effect of its changing vertical optical depth, τg with time of the day (or SZA or airmass) in equation (1) on F0.
 We assumed for simplicity that the vertical optical depth, τg changes linearly with airmass, m (which is equivalent to τg changing with time) during the measurements period so that
where τ0 is the total vertical atmospheric optical depth when m=m0, m is the airmass at a given time, m0 is the lowest airmass where measurements for a particular day were taken, and k determines how quickly τg changes with airmass. The error in this linear assumption at 95% confidence interval is less than 0.5% for both days, which is relatively small and hence is neglected as an error source here.
 With this assumption,
 As an illustration, consider a situation where the maximum water vapor amount changes by 20% during the period of measurements (i.e., between maximum m and m0). Considering wavenumbers at which τg is dominated by water vapor, this allows us to obtain expressions for the parameter k in terms of m.
 For a 20% change in water vapor amount, k is given by
where Δm is the change in airmass and depends on the measured minimum and maximum solar zenith angles of interest. On 22 August 2008, Δm≈2.1, so that the range of k values is − 0.1 ≤ k ≤ 0.1.
 With this range of values of k, different values of τ0 between 0.01 and 1.0 were chosen and τg was calculated using equation (4). When τ0 < 0.1, it is considered that the calculations are in atmospheric microwindows and when τ0 > 0.1, the calculations are considered to be near line centers. The Langley method was applied to calculate a value of F0, if τg = τ0 at all times and for τg using equation (4), which is denoted F0new for a range of k values. The difference between F0 and F0new indicates the effect of changing water vapor amount on the derived ESS.
 Figure 2 shows the relative difference (F0new − F0)/ F0new, for a range of values of τ0 and k. The percentage difference between F0 and F0new increases as τ0 increases. At low τ0 (0.01 up to about 0.05, typical values in atmospheric microwindows in near-IR), the difference between F0 and F0new is about 1% or less, but for higher τ0 (0.2 up to about 1.0, which are more typical near line centers), this difference is up to about 60%. There is some asymmetry between the case of positive k (τg increasing with m) and negative k (τ decreasing with m) because of the logarithmic nature of the calculations.
 In conclusion, the ESS derived from Langley analyses at atmospheric line centers are less reliable than those in atmospheric microwindows when the amount of water vapor is changing during measurements. Using m and k values appropriate for 18 September 2008 observations, the difference between F0new and F0 obtained was up to about 70% near line centers for a 20% change in water vapor amount. This is due to the fact that the observations were made at higher airmasses on 18 September 2008 than 22 August 2008. Using the GPS water vapor column measurements given above, the relative difference between F0new and F0 was lower at atmospheric line centers on 18 September 2008 than on 22 August 2008, indicating that the ESS for 18 September 2008 is more reliable than 22 August 2008. Note that the water vapor amount decreased from m0 to m, thus giving a negative k for both days.
 As a result of the above calculations, spectral regions where changes in the water vapor amount could greatly affect our derived solar spectrum were filtered out as described in section 3.3.2. A correction of less than 1%, which is appropriate for microwindows, is negligible compared to other uncertainties, notably the uncertainty from the FTS measurements and calibration (see section 3.2.3).
22.214.171.124 The Effect of Changing Aerosol Amount
 Using measurements with a handheld Solar Light Microtops II Sun photometer during the field campaign at 5 wavelengths (380, 440, 675, 935, and 1020 nm), we observed no systematic variation in the aerosol optical depth (AOD) with airmass during the period of measurements on both days. During the observation period, the AOD at 675 nm varied between 0.07 and 0.12 on 22 August 2008 and between 0.07 and 0.14 on 18 September 2008. At 1020 nm, the AODs were lower, varying between 0.03 and 0.08 on 22 August 2008 and between 0.04 and 0.08 on 18 September 2008. We do not have AOD information at the wavelengths relevant here. Thus, we would have to make an extrapolation (such as Ångström's extrapolation [Iqbal, 1983]) to estimate the AOD variation. We note that if there was any systematic variation in AOD with airmass, the methodology used for water vapor in section 126.96.36.199 could be applied and used as correction on the derived ESS. However, the impact of AOD variations is considered to be part of the statistical errors on the Langley fits.
3.2.6 Quantified Uncertainty on the CAVIAR Extraterrestrial Solar Spectrum
 The quantified error in the derived ESS is the root-sum-square of the solar FTS measurement and the statistical errors described above. The total estimated (k = 1, 67% confidence interval) uncertainties in the derived ESS varied with wavenumber from about 3.4% to 6.0% for 22 August 2008 and from about 3.3% to 5.9% for 18 September 2008. The ESS uncertainty from 18 September 2008 observations is similar to the FTS calibration uncertainty alone because the statistical errors from this day are quite small (0.5% at k = 2). The statistical errors from 22 August 2008 are higher and increase the uncertainty of the derived ESS from this day.
3.3 Filtering of the CAVIAR Extraterrestrial Solar Spectrum
 Filtering of the derived ESS is essential to obtain as much reliable information as possible, especially from spectral regions close to the band centers. The top panel of Figure 3 shows an example of the derived ESS in a 20 cm−1 spectral region on 18 September 2008 when no filtering has been applied. It can be immediately seen from this figure that some of the results, for example, at about 5690, 5692, and 5698 cm−1 are not physical, because the spikes and dips seen at these wavenumbers do not represent emission from the Sun. Thus, there is the need to filter out such results. The following criteria have been used to filter the derived ESS.
3.3.1 Threshold Irradiance and Range of Airmass
 Langley fits were only used when the spectral irradiance measured by the FTS is above a threshold value of 0.0015 W m−2/cm−1. This ensured that fitting was not done on spectral irradiance values which may be considered as “noise”, i.e., in regions of high atmospheric absorption.
 In rejecting some FTS spectral irradiances as noise, a situation could arise where good observations cover only a small range of airmass for the Langley fits. This is not desirable because, as discussed in section 3.2.1, when airmass is changing slowly with time, variation in the atmospheric conditions that may affect the Langley analyses will become significant [Harrison and Michalsky, 1994]. Thus, a criterion on the range of airmass that should be used was put in place to complement the threshold irradiance criterion.
 For the 22 August 2008 observations, using the range of observed SZA, the minimum difference between the minimum and maximum airmasses was chosen to be 0.9616 for the Langley fits to be used. This difference ensured that the SZA used for any Langley extrapolation for this day must change by at least about 24°. Similarly, for the 18 September 2008 observations, the difference in airmasses was chosen to be 1.2583, i.e., the change in the SZA must not be less than about 20°.
 The derived ESS is shown in the bottom panel of Figure 3, when these filtering criteria were applied, which removes the unphysical results shown in the top panel.
3.3.2 Filtering of Water Vapor Lines Due to Changing Water Vapor Amount
 As shown in section 3.2.5, changes in water vapor amount can lead to significant errors in the derived ESS near line centers and hence must be filtered out. To do this, the line-by-line code of Mitsel et al.  was used to calculate water vapor optical depths using a vertical water vapor profile from radiosonde launches at Camborne with 62 vertical levels, the HITRAN 2008 molecular spectroscopic database [Rothman et al., 2009], and the MT_CKD 2.5 water vapor continuum model [Mlawer et al., 2012]. The resolution of the line-by-line code was 0.001 cm−1 and the output step was 0.03 cm−1. This resolution is small enough to resolve tropospheric water vapor lines.
 Langley fits were not used at wavenumbers where line-by-line simulated water vapor optical depth was more than 0.2 to exclude the less reliable fits as explained in section 188.8.131.52. Ideally, water vapor optical depths typical in atmospheric microwindows, which is about 0.05 or less, would have been used as a cut-off value (see section 3.2.5 and shown graphically in Figure 2). However, by adopting such low optical depths threshold, there was a greater chance of filtering out some solar lines, which were masked, in some spectral region. Note that the filtering criteria discussed in section 3.3.1 were applied first before the filtering of any water vapor lines is done.
 Choosing the different filtering criteria to obtain meaningful results required a number of compromises. Using strict criteria led, in some cases, to the loss of useful data. On the other hand, when the filtering criteria are relaxed, it may lead to results that are not entirely reliable. The final filtering criteria were chosen in such a way as to minimize any of these effects.
 Typical Langley plots at atmospheric line centers and microwindows for 18 September 2008 are shown in the upper and lower panels of Figure 4, respectively. As expected, spectral irradiance decreases more rapidly with increasing airmass within the line centers than in the microwindows because of the higher optical depth.
 The ESS derived for both days have been corrected to account for the elliptical nature of the Earth's orbit by using the following expression:
where dn is the day number of the year, Fext(dn) is the derived ESS at any day dn and is the derived ESS corrected to 1 astronomical unit (1 AU) [Liou, 2002]. Because this expression is derived from the known value of the eccentricity of the Earth's orbit (neglecting small terms in the square of the eccentricity), it is accurate to about 0.1%.
 In this section, will be presented to enable a comparison with other ESS. We concentrate on 18 September 2008 because, as discussed in section 3.2, the Langley analysis for this day is judged more reliable.
 The spectral structure in Figure 1 has been discussed in section 1. Figure 5 shows the CAVIAR ESS together with the Kurucz-observed ESS, Kurucz-modeled ESS, and Thuillier ESS in a 10 cm−1 spectral region. It can be seen that there is a disagreement between some solar line positions and strengths of the CAVIAR ESS and Kurucz-modeled ESS. However, these disagreements are not as big as those seen in Figure 1. It is also clear from Figure 5 that the solar line strengths and positions of the Kurucz-observed ESS and CAVIAR ESS agree well, although there is a slight wavenumber shift of about 0.01 cm–1 between them. There is also an agreement between the solar line positions and strengths of some strong lines in the CAVIAR ESS, Kurucz-modeled ESS, and Kurucz-observed ESS. For example, we see this agreement at about 6287, 6292.1, 6293.8, and 6295.8 cm–1. Due to its coarse spectral resolution, the Thuillier ESS shows only hints of any structure in this and other spectral regions. The agreement in solar line positions and strengths between the CAVIAR ESS and two independent observational ESS (ACE-FTS ESS and Kurucz-observed ESS) shows that the measurements and methods used in deriving our solar spectrum are robust. This agreement also demonstrates that there is a greater confidence in the representation of solar line positions and strengths by the CAVIAR ESS than in the widely used Kurucz-modeled ESS. In this spectral region, there is a 6% wavenumber average difference between the CAVIAR ESS and Kurucz-modeled ESS, an 8% difference between the CAVIAR ESS and Thuillier ESS, and an 11% difference between the CAVIAR ESS and Kurucz-observed ESS. The 3% difference between the Kurucz-observed ESS and Thuillier ESS is well within the uncertainties of both solar spectra (see section 1). This agreement is not surprising because the calibration of the Thuillier ESS has been partly used for the calibration of the Kurucz-observed ESS [Kurucz, 2008]. In other spectral regions, where a comparison between the CAVIAR, Kurucz-modeled, Kurucz-observed, and Thuillier ESS was possible, there were similar results in the representation of solar line positions and strengths. The differences between their absolute background levels vary with wavenumber.
 Figure 6 shows the CAVIAR ESS together with the Fontenla, Kurucz-observed, and Kurucz-modeled ESS in a 10 cm–1 spectral region. There is a small gap at about 5963.6 cm–1 in the CAVIAR ESS, due to the filtering process described in section 3.3. The Fontenla ESS has much less structure than the other three spectra and has just two strong absorption solar lines in this spectral region. The positions of the strongest lines in this spectral region are well represented by all four spectra, but they are much stronger in the Fontenla ESS. Once more there is a good agreement between the solar line positions and strengths of the CAVIAR ESS and Kurucz-observed ESS (again with a slight wavenumber shift). The disagreement between the solar line positions and strengths of the CAVIAR ESS and Kurucz-modeled ESS can also be seen. The lack of a detailed structure of the solar spectrum by the Fontenla ESS as compared to the CAVIAR, Kurucz-observed, and Kurucz-modeled ESS was found in other spectral regions where a comparison between the four spectra was possible. The results given in Figures 1, 5, and 6 show the superiority of the solar spectra derived from measurements as compared to those derived from models. In Figure 6, there is a 5% difference between the CAVIAR ESS and both Fontenla ESS and Kurucz-modeled ESS and a 9% difference between the CAVIAR ESS and Kurucz-observed ESS. We also see from Figure 6 that the Fontenla ESS and Kurucz-modeled ESS are almost the same in their absolute levels, although they are independently derived models of the solar spectra.
 In the spectral region covered by our analysis (2000–10,000 cm–1), the absolute level of CAVIAR ESS is up to about 10% lower than the Fontenla ESS and Kurucz-modeled ESS. A lower difference (1–6%) was obtained in the spectral region where there is more confidence in the FTS calibration (between about 4000 and 6500 cm−1or 1.5–2.5 µm). The absolute level of the CAVIAR ESS is about 7–8% lower than the Thuillier ESS in this spectral region. This difference increases to more than 10% in spectral regions where there is less confidence. Figure 7 shows the CAVIAR ESS and Thuillier ESS with their uncertainties (k = 2, 95% confidence interval) in two spectral regions; one where the confidence in the calibration of our FTS is higher (top panel) and one where the confidence is less (bottom panel). These two regions are representative of the observed differences between these spectra from 4200 to 10,000 cm–1 (1.0–2.4 µm). The top panel shows that the absolute levels of these two solar spectra do not agree even within their uncertainties. However, the bottom panel shows that their absolute levels agree within their uncertainties, but this is due to the fact that the uncertainties of the CAVIAR ESS are larger in this spectral region. This evidence indicates that the absolute levels of the CAVIAR ESS and Thuillier ESS are significantly different.
 We note that the ESS obtained by Harder et al.  (called SIM ESS here) was up to about 8% lower than the Thuillier ESS in the near-IR, although Harder et al.  believe that there is a systematic bias in the SIM ESS. The implication of this 8% difference between the Thuillier ESS and the CAVIAR ESS will be discussed in section 5.
 The extraterrestrial solar spectra (normalized to the distance of 1 AU) derived from the observations from both 22 August 2008 and 18 September 2008 are expected to be the same despite the fact that there was about a 4 week gap between them. This is because these observations were made close to the solar minimum when there are no sunspots. The three panels of Figure 8 show the CAVIAR ESS on these two days in three 10 cm−1 spectral regions. Immediately evident is the fact that the solar line positions and strengths are the same for both spectra. This agreement was also seen in other spectral regions. Thus, the method used here to derive the spectral structure of the ESS from ground-based measurements appears robust in this regard. Figure 8 also shows that the ESS derived for 18 September 2008 is higher than for 22 August 2008. There is a 2%, 5%, and 7% difference between the spectra from these two days in the top, middle, and bottom panels, respectively. Generally, from 2000 to 10,000 cm–1, the difference between these two solar spectra is lower (2–5%) in the spectral region where there is more confidence in the calibration of the FTS, increasing up to about 10% where the confidence is lower.
 The uncertainties (k = 1, 67% confidence interval) associated with the CAVIAR ESS for both days in these three 10 cm–1 spectral windows are also shown in Figure 8. Their absolute levels agree within their uncertainties in these three spectral regions. Although the measurements on the two days agree within their k = 1 uncertainties, based on the discussions in section 3.2, we believe that the ESS derived for 18 September 2008 is more reliable.
5 Merging and Normalization of the CAVIAR Extraterrestrial Solar Spectrum
 The CAVIAR ESS was derived in atmospheric windows between 2000 cm–1 and 10,000 cm–1 as stated in section 3.1. The top panel of Figure 9 shows this spectrum. The spectral gaps, corresponding to water vapor absorbing bands where FTS measurements were not possible, can be clearly seen. Because of this limitation, the CAVIAR ESS cannot be generally used for radiative transfer calculations. To produce an ESS that can be used more generally, we used other high-resolution extraterrestrial solar spectra to fill some of the gaps in the CAVIAR ESS. Data from these sources were only used at wavenumbers where the CAVIAR ESS has no data. We chose only to do this for wavenumbers greater than 4000 cm–1, where such gaps were relatively narrow in wavenumber space. From 4000–4400 cm–1, data from the ACE-FTS ESS was used. From 4761–5208 cm–1 and 5813–6410 cm–1, data from the Kurucz-observed ESS were used. Finally, the Kurucz-modeled ESS was used in any other spectral region not covered by the CAVIAR, ACE-FTS, or Kurucz-observed ESS, that is, from 4400 to 4761, 5208 to 5813, and 6410 to 10,000 cm–1.
 To merge these spectra, the CAVIAR ESS is multiplied by a wavenumber dependent factor, f, to take it to the level of either the ACE-FTS ESS, the Kurucz-observed ESS, or the Kurucz-modeled ESS, i.e.,
where is either the ACE-FTS ESS, the Kurucz-observed ESS, or the Kurucz-modeled ESS and is the CAVIAR ESS.
 After multiplying the CAVIAR solar spectrum by f, the gaps in the CAVIAR ESS were filled with spectral irradiances from either the ACE-FTS, Kurucz-observed, or Kurucz-modeled ESS depending on the wavenumber range. The interpolated CAVIAR ESS, with data at all wavenumbers between 4000 and 10,000 cm–1, was then brought back to its initial level by dividing by the same factor, f. Note that this procedure works for the ACE-FTS ESS even though it has no absolute calibration. The factor f is calculated using the ACE-FTS ESS “transmittance.” The advantage of the technique is that it uses the structure observed in the ACE-FTS ESS (and the structure of the other 2 spectra), while using the absolute calibration of the CAVIAR ESS. The merged CAVIAR ESS is shown in the bottom panel of Figure 9.
 It is interesting that from Figure 9 the derived CAVIAR ESS shows a distinctive dip near 7800–7900 cm–1 (near 1.27 µm). We initially thought that this feature was due to, for example, contamination by O2 in the Earth's atmosphere. However, the intensity of this signature is too strong to be explained by O2. The O2-profile and column amount in the Earth's atmosphere is rather stable (comparable with variation in the atmospheric pressure at surface), and so, most part of the collision-induced O2 absorption in the atmosphere should be removed from the resulting ESS by the Langley approach. Thus, we suggest that this fall is intrinsic to the extraterrestrial solar emission at these wavenumbers. We note that the Thuillier ESS also indicates a dip at about 7800 cm–1 but is less marked due to its lower spectral resolution. We also observe some spikes in the CAVIAR ESS in the spectral region 8000–9500 cm–1. These spikes are from our measurements and appear in this weak water vapor band only because they have been filtered out in strong water vapor bands.
 In Figure 10a a running average through 15 cm–1 is applied to the Thuillier, Kurucz-modeled, Fontenla, and CAVIAR ESS. The spectral shapes of the Kurucz-modeled and Fontenla spectra are almost the same but are clearly different from the Thuillier ESS or CAVIAR ESS. Beyond about 7000 cm–1, the Thuillier ESS flattens out, both the Kurucz-modeled and Fontenla spectra keep rising, and the CAVIAR ESS starts to fall slightly. Figure 10b shows the CAVIAR and Thuillier ESS, with their k = 2 uncertainties from 4200 to 10,000 cm–1. The increase in the overlap of their uncertainties with wavenumber depicted in Figure 7 can be clearly seen from this figure. Given the increase in uncertainty in the CAVIAR ESS at higher wavenumbers, it is not possible to assess whether the flattening out of the CAVIAR ESS, relative to the other ESS, is a robust feature. As the Thuillier ESS is often used as a reference ESS, we have also normalized the CAVIAR ESS to this solar spectrum over the sub-band between 4200 and 10,000 cm–1 where there is an overlap between the two. The integrated solar spectral irradiances are 341W m–2 and 372 W m–2 for the CAVIAR and Thuillier spectra, respectively. The difference between these two irradiances is 31 W m–2 or 8.3%, and hence the CAVIAR ESS must be increased by this fraction to agree with the Thuillier ESS. If the CAVIAR ESS is correct, then there is about 8% less solar irradiance between 4200 and 10,000 cm–1 than is currently thought. Because the total solar irradiance (TSI) is relatively well known, this “loss” in near-IR irradiance must be compensated by an increase in other wavelengths regions. However, it should be noted that the CAVIAR ESS derived from only 1 day of measurement has been used for this inference and it would clearly be desirable to have more analysis days to improve confidence. In addition, the Thuillier analysis assumed a TSI value of 1381 W m–2 and it is now known that 1360.8 ± 0.5 W m–2 is a more reliable value for the TSI [Kopp and Lean, 2011]. If the Thuillier ESS is scaled by using this lower value, the integrated irradiance over the region of interest is reduced to 367 W m–2, and the difference is reduced from 31 to 26 W m–2 (or from 8.3 to 7%).
 Thus, the CAVIAR ESS can be regarded as a composite high-resolution ESS, with more than 70% of it derived from measurements. Depending on individual preferences, either the CAVIAR ESS or the normalized (to the Thuillier) CAVIAR ESS can be used for line-by-line radiative transfer calculations (and other calculations that require a detailed spectrally resolved ESS). Both sets of data are given in the auxiliary material.
6 Discussion and Conclusions
 A unique set of high spectral resolution ground-based measurements has been used to derive the ESS in the near-IR using the Langley method. These measurements were made using a Sun-pointing FTS at different solar zenith angles during the Camborne field campaign in summer 2008. This ESS has been derived mainly in the atmospheric windows between 2000 and 10,000 cm−1 (1–5 µm) because of saturation of the FTS at atmospheric band centers. The CAVIAR ESS has been derived using observations from two field campaign days: 22 August 2008 and 18 September 2008, with the latter deemed more reliable.
 The CAVIAR ESS has been compared with five other extraterrestrial solar spectra: the widely used theoretically derived Kurucz ESS, the space-based measured ACE-FTS ESS, the mostly space-based composite (but relatively low spectral resolution) Thuillier ESS, the observed Kurucz ESS from FTS measurements, and the Fontenla ESS obtained using semiempirical models.
 Comparison of the CAVIAR ESS with the observed ACE-FTS and Kurucz high-resolution spectra confirms that, in the region of overlap, the CAVIAR ESS represents the positions and strengths of solar absorption lines accurately. This gives us confidence to apply the technique in other spectral regions. The solar absorption line positions and strengths by the Kurucz-modeled and Fontenla spectra are often quite poorly represented. There is also an agreement between the solar line positions of the CAVIAR and Kurucz-observed spectra. One plausible reason why the Kurucz-modeled and Fontenla spectra have a poorer representation of solar line positions and strengths is that the models used for deriving them do not fully represent the Sun's atmosphere or all the possible spectroscopic transitions in the solar atmosphere. Hence, some caution should be taken when using either the Kurucz-modeled or Fontenla spectra especially for line-by-line calculations to avoid possible artifacts, such as those obtained by Tallis et al. .
 The absolute level of the CAVIAR ESS shows some differences with that of the Kurucz-modeled, Kurucz-observed, Fontenla and Thuillier spectra. The CAVIAR ESS is consistently lower in absolute level than these four solar spectra in most of the spectral region between 2000 and 10,000 cm−1. The CAVIAR ESS is typically 6% (or less) lower than the Kurucz-modeled and Fontenla spectra in the spectral regions where there is more confidence in the calibration of the FTS; in the spectral region where the confidence in the calibration is low, this difference can be up to 10%.
 The absolute level of the CAVIAR ESS has also been compared with the lower-resolution Thuillier ESS. The CAVIAR ESS can be about 8% lower than the Thuillier ESS in spectral regions where there is a greater confidence in the calibration of the FTS. The CAVIAR ESS and Thuillier ESS generally do not overlap within their known k = 2 uncertainties, indicating a significant difference between the two measurement systems. We are unable to state which is more reliable. Integrated over the 4200 cm–1 and 10,000 cm–1 (1 to 2.4 µm) spectral region, the difference between the CAVIAR ESS is 8.3% lower than the Thuillier ESS, although this difference is reduced to 7% if the Thuillier ESS is scaled by more recent values of the total solar irradiance. If the CAVIAR ESS is correct, then there is a 7% “loss” of solar irradiance in the 4200–10,000 cm–1 region. Because the TSI is well constrained [Kopp and Lean, 2011], the “loss” of solar irradiance from the near-IR would need to be compensated by a gain at other wavenumbers. This emphasizes the need for further well-calibrated measurements of the near-IR ESS, from both satellites and the surface. In particular, it would be desirable to repeat the ground-based Langley analysis using measurements for a wider range of atmospheric conditions, to further assess whether the uncertainty analysis is indeed robust, for example, to the presence or absence of thin cirrus.
 Within the limit of the combined (k = 1, 67% confidence interval) uncertainty the CAVIAR ESS derived from observations of 22 August and 18 September 2008 agree with each other in the 2000–10,000 cm–1 spectral region.
 To produce an ESS in the near-IR spectral region (that can be used, for example, for line-by-line radiative transfer calculations including band centers where we are unable to observe), three other high-resolution solar spectra; the ACE-FTS, Kurucz-observed, and Kurucz-modeled spectra have been used to fill in the missing data in the CAVIAR ESS. We have also produced a version of the CAVIAR ESS, which has been normalized to the Thuillier ESS, because this solar spectrum is widely accepted as a reference solar spectrum.
 CAVIAR was funded by the UK's Natural Environment Research Council (NERC) and Engineering and Physical Sciences Research Council (EPRSC). Kaah P Menang would like to thank the University of Reading for funding his Ph.D. research. We thank Liam Tallis for his work in helping to make these measurements, and for his input and advice on the Langley method during the initial stages of this work. The reviewers made many very insightful comments that improved the manuscript.