Corresponding author: W. Greuell, Earth System Sciences group, Wageningen University and Research Centre, Lumen Building 100, Droevendaalsesteeg 3, NL 6708 PB Wageningen, Netherlands. (email@example.com)
 This paper discusses Surface Insolation under Clear and Cloudy skies derived from SEVIRI imagery (SICCS), a physics-based, empirically adjusted algorithm developed for estimation of surface solar irradiance from satellite data. Its most important input are a cloud mask product and cloud properties derived from Meteosat/Spinning Enhanced Visible and Infrared Imager (SEVIRI) observations. These observations set the characteristics of the output, namely, a temporal resolution of 15 min, a nadir spatial resolution of 3 × 3 km2, the period from January 2004 until at least November 2012, and the domain equal to most of the Meteosat disc. SICCS computes global, direct, and diffuse irradiance separately. Direct irradiance for cloudy skies is estimated with an empirical method. Hourly means retrieved with SICCS were validated with data from eight Baseline Surface Radiation Network stations for the year 2006. We found median values of the station biases of +6 W/m2 (+5%) for direct irradiance, +1 W/m2 (+1%) for diffuse irradiance, and +7 W/m2 (+2%) for global irradiance. Replacing the three-hourly aerosol optical thickness input by monthly means introduces considerable additional biases in the clear-sky direct (−6%) and diffuse (+26%) irradiances. The performance of SICCS does not degrade when snow covers the surface. Biases do not vary with cloud optical thickness and cloud particle radius. However, the bias in global transmissivity tends to decrease with increasing cloud heterogeneity, and the bias in direct transmissivity is a function of the solar zenith angle. We discuss why satellite retrieval of surface solar irradiance is relatively successful.
If you can't find a tool you're looking for, please click the link at the top of the page to "Go to old article view". Alternatively, view our Knowledge Base articles for additional help. Your feedback is important to us, so please let us know if you have comments or ideas for improvement.
 Quantification of solar radiation at the Earth's surface is useful for several reasons. It is needed for the calculation of evapotranspiration, plant growth, and soil moisture, which affect crop yield and the water budget. Also, surface solar radiation can be assimilated into numerical weather prediction (NWP) models [Rodell et al., 2004], and it can be exploited for the evaluation of climate models [Freidenreich and Ramaswamy, 2011]. However, the largest socioeconomic benefit of knowledge of surface solar radiation probably resides in its use for the estimate of the yield of solar heating and power systems [Arvizu et al., 2011].
 For all these purposes, temporally and spatially complete fields of surface solar radiation are desired, which can be derived by three categories of methods, namely, (i) by interpolation of ground-based measurements [Šúri et al., 2005], (ii) by estimation from satellite observations [Bishop et al., 1997; Gupta et al., 1999; Deneke et al., 2008; Dürr and Zelenka, 2009; Müller et al., 2009; Wang and Pinker, 2009; Wang et al., 2011b; Huang et al., 2011] and (iii) by reanalyses with NWP or climate models [Zib et al., 2012]. Moreover, fields produced by different methods can be combined [Journée and Bertrand, 2010]. Interpolation of ground-based measurements has two major drawbacks. First, compared with the spatial density of satellite pixels, the density of ground-based stations is generally much lower. Second, ground-based measurements are prone to measurement errors, which can be considerable if sensors are of lower quality and/or checks and maintenance of the instruments are infrequent. Reanalyses with NWP and climate models may also suffer from inadequate spatial resolution of the computed fields. More important is that these calculations depend on model simulations of the factor that dominates the calculation of atmospheric transmission of solar radiation, namely, clouds. Errors in the simulation of the temporal and spatial distribution of clouds cause large errors in solar radiation calculations. Satellite retrievals of surface solar radiation, on the other hand, may have high spatial resolution, depending on sensor resolution, and the timing of cloud fields is not an issue, provided the interval between subsequent images is short enough.
 This paper discusses a method for the estimation of surface solar radiation from satellite observations, namely, version 2 of surface insolation under clear and cloudy skies (SICCS) from Meteosat SEVIRI imagery, which was built on version 1 [Deneke et al., 2008]. The method was applied to data from Meteosat's Spinning Enhanced Visible and Infrared Imager (SEVIRI), which resulted in great temporal (15 min) and considerable spatial (3 × 3 km2 at nadir) resolution of the product. Validation studies will provide evidence of the high quality of the data set.
 The previous version of SICCS solely dealt with the calculation of global irradiance (also called surface solar irradiance), defined as the amount of downwelling solar energy incident on a horizontal plane at the Earth's surface and integrated over the entire solar spectrum. However, in view of the importance of calculations of solar radiation impinging on solar panels, it is necessary to enable the computation of the flux of solar radiation through an upward-facing surface with any orientation and tilt [see Klucher, 1979]. This requirement can be fulfilled if global irradiance is decomposed into two components, namely, direct irradiance—that part of global irradiance that was transmitted directly through the atmosphere to the Earth's surface without interaction with atmospheric components, and diffuse irradiance—that part of global irradiance that reaches the surface after at least one scattering event in the atmosphere. The algorithm described here includes the calculation of direct and diffuse irradiance.
 In its calculations of the irradiances SICCS considers, apart from the solar zenith angle (SZA), all of the relevant variations in atmospheric constituents and surface properties, namely, cloud optical thickness (COT), cloud particle radius, cloud phase, aerosol optical thickness (AOT) at 500 nm, the Ångström exponent (AEXP), the aerosol single scattering albedo (SSA), surface elevation, visible and near-infrared surface albedo, and integrated water vapor (IWV), exploiting state-of-the-art input fields of these variables. The calculations of atmospheric transmission are performed with a detailed radiative transfer model, which makes SICCS a physics-based algorithm. There are two empirical edges to the algorithm, which are the calculation of direct irradiance for cloudy pixels and a correction to the global transmissivity of ice clouds.
 The cloud properties exploited by SICCS are computed with the clouds physical properties (CPP) algorithm from SEVIRI data [Roebeling et al., 2006]. As stated before, clouds form the single most important factor among those that determine the accuracy of the product. Therefore, our product has the temporal and spatial resolution of the CPP/SEVIRI cloud products. Also, the SICCS and the CPP algorithm rely on the same radiative transfer model. Thus, CPP is closely tied to SICCS.
 While SICCS is operated at the Royal Netherlands Meteorological Institute (KNMI), there exist at least three other surface solar irradiance products based on SEVIRI data that are available from other institutions. These are as follows:
The operational Satellite Application Facility on Climate Monitoring (CM-SAF; see Schulz et al., 2009) product [Müller et al., 2009].
 The algorithm is described in section 2 and the input data sets in section 3. Section 4 deals with the ground-based measurements from eight Baseline Surface Radiation Network (BSRN) stations used for the validation. Section 5 details some important aspects of SICCS. The validation itself forms the topic of section 6. Section 7 summarizes this paper, and finally section 8 discusses why satellite retrieval of surface solar irradiance is generally quite successful and what limits the performance of the algorithms.
 SICCS can be considered as consisting of two parts: (1) the input describing the state of the atmosphere and the surface and (2) the algorithm itself, which consists mainly of radiative transfer calculations based on the input data and aimed at the computation of the surface irradiances. This section deals with the algorithm itself of the second version of SICCS. A detailed description of version 1 can be found in the study of Deneke et al. . Here, we restrict the text to an outline of the algorithm while treating modifications with respect to version 1 more extensively.
 Before the application of the algorithm, radiative transfer calculations were performed to create lookup tables (LUTs). The LUTs contain transmissivities on a multidimensional grid spanned by discrete values of the variables describing the atmosphere and the surface. We will refer to the ratio of surface to the top-of-the-atmosphere (TOA) downwelling solar irradiance as global transmissivity and refer to the components of global transmissivity corresponding to direct and diffuse irradiance as direct and diffuse transmissivity. The radiative transfer calculations were performed with the doubling adding KNMI (DAK) radiative transfer model, which will be described in section 5.1. In total, eight LUTs were produced, as follows:
Four LUTs for clear skies: two LUTs with global transmissivities, that is one for visible wavelengths (VIS: 240–704 nm) and one for near-infrared wavelengths (NIR: 704–4606 nm), and two LUTs with direct transmissivities in the same wavelength bands. The LUTs were created for the two mentioned bands because the surface albedo input is given in these bands. Within the VIS and the NIR band, the surface albedo, the aerosol SSA, and the aerosol asymmetry parameter are assumed to be spectrally constant (see section 3). Spectral variations in aerosol extinction within the two bands are prescribed by the AOT at 500 nm and the AEXP and are taken into account by DAK. DAK also calculates spectral variations in transmissivity due to atmospheric components like cloud particles and water vapor in detail (see section 5.1). The clear-sky LUTs are seven-dimensional, with the following independent variables along the axes: AOT at 500 nm, AEXP, aerosol SSA, surface elevation, surface albedo, IWV, and cosine of the solar zenith angle (COSSZA). Table 1 contains the discrete values of the independent values for which the calculations were made.
Two LUTs for atmospheres with water clouds giving global transmissivity in the VIS and NIR, respectively. The water-cloud LUTs are five-dimensional, with COT, effective droplet radius, surface albedo, IWV, and COSSZA spanning the axes. Discrete values of these variables used to construct the LUTs are given in Table 1.
Two LUTs for atmospheres with ice clouds giving global transmissivity in the VIS and NIR, respectively. The ice-cloud LUTs are conceptually identical to the water-cloud LUTs but effective droplet radius is replaced by effective crystal radius.
Table 1. Specifications of the LUTs Created with DAK, the Variables Spanning the Axes of the LUTs, and the Discrete Values of Those Variables
Cosine of the solar zenith angle
From 0.2 to 1.0 in steps of 0.05
0.0, 0.5, and 1.0
Integrated water vapor
(0.2, 0.6, and 1.0) × 29.61 mm
Aerosol optical thickness at 500 nm
From 0.0 to 0.8 in steps of 0.1
0.2, 1.0, and 1.8
Aerosol single scattering albedo
from 0.8 to 1.0 in steps of 0.04
0, 1000, and 2000 m above sea level
Cloud optical thickness
From 0.25 to 256 in multiplicative steps of √2
Effective droplet radius (water clouds)
1, 3, 5, 8, 12, 16, and 24 µm
Effective crystal radius (ice clouds)
6, 12, 26, and 51 µm
 The algorithm is then applied to each SEVIRI pixel in each image and proceeds as follows:
1.A cloud mask is constructed from the SEVIRI reflectances and brightness temperatures [see Roebeling et al., 2008]. The cloud mask procedure results in one of three possible qualifications, namely, clear, cloud contaminated, or cloudy. Clear-sky pixels are then treated according to step 2 and the other pixels according to steps 3 and 4.
2.If a pixel is found to be clear in step 1, the four clear-sky transmissivities (both global and direct in the VIS and in the NIR) are computed. Because the elements of the LUTs provide the transmissivities only for a limited number of discrete values of the input parameters while the input parameters themselves vary on continuous scales, the transmissivities must be found by interpolation. In five of the seven dimensions transmissivities are linearly interpolated. In the remaining two dimensions (albedo and the water vapor), correction equations that account for multiple reflection and variations in water vapor absorption are applied. Coefficients in these correction equations are calculated from the elements of the LUTs [see Deneke et al., 2008].
3.If a pixel is found to be cloudy or cloud contaminated in step 1, cloud phase, COT, and cloud particle effective radius are determined with the algorithm of cloud physical properties (CPP). CPP uses the SEVIRI reflectances at 0.64 and 1.63 µm and the brightness temperature at 10.8 µm to retrieve the mentioned cloud properties. Hollmann et al.  report on validation of CPP products. Because CPP exploits measured radiation to compute cloud properties and SICCS itself exploits cloud properties to compute radiation, the two algorithms are to some extent each other's inverse. Therefore, important elements of CPP and SICCS were treated in a consistent manner. The radiative transfer calculations are carried out with the same radiative transfer model (DAK) on almost the same LUT grid. Single scattering calculations for water droplets and ice crystals are performed with the same models. The same assumptions about the clouds are made, mainly that of horizontally and vertically homogeneous clouds, assumptions about cloud height and thickness, and assumptions about droplet radius distribution. Also, the state of the rest of the atmosphere is identical, which means, among others, cloudy atmospheres are assumed to be free of aerosol. In the case of cloudy atmospheres, both CPP and SICCS have the surface at sea level. Moreover, CPP and SICCS exploit the same surface albedo product (see section 3) as background. Despite these similarities, there are also important differences. In CPP, radiative transfer calculations are monochromatic; in SICCS, broadband calculations are performed. The CPP LUTs have two dimensions in addition to the five dimensions of the SICCS cloud LUTs, namely, the view zenith angle and the relative azimuth angle. Finally, CPP takes radiance measurements as input, whereas SICCS produces irradiances. For more detailed information about CPP, we refer to Roebeling et al.  and Deneke et al. .
4.Given the CPP cloud properties, the global transmissivities in the VIS and the NIR are determined with the cloud LUTs, using the same interpolation and correction methods as in step 2 for clear skies. In case CPP retrieves a COT smaller than the lowest value spanning the cloud LUTs (0.25), the pixel is yet considered to be clear and transmissivities are calculated with the clear-sky LUTs (step 2).
5.Broadband (240–4606 nm) transmissivities are computed from the transmissivities in the VIS and the NIR, weighing the VIS and the NIR transmissivities by the fractional contributions of VIS and NIR to the incoming radiation at the TOA (0.478 and 0.522, respectively). This step yields the broadband global transmissivity for all pixels and the broadband direct transmissivity for the clear pixels. To keep the terminology short, we will from now on omit the adjective “broadband.”
6.To correct for a bias found during initial validations (see section 6.3), the global transmissivity for ice clouds is diminished by 0.0375. Resulting negative values are set equal to zero.
7.For cloudy atmospheres, direct transmissivity (Tdir) is computed from clear-sky direct transmissivity (Tdir,clr) and COT (τcld) with the following equation:
where A = 3.35 and τcld,sc = 7.81 for water clouds and A = 3.96 and τcld,sc = 3.86 for ice clouds (see section 5.2).
8.Gaps in the time series are filled. Two types of gaps exist. The first type is a consequence of the limitation of the calculations to SZAs less than 78°. This limitation was introduced because errors in the retrieved CPP cloud properties increase strongly at large SZAs, which is due to the increasing influence of three-dimensional cloud effects and due to the curvature of the Earth. As in SICCS version 1 [Deneke et al., 2008], global transmissivity for SZA beyond the limit of 78° (Tglob,lowsun) is estimated with the following equation:
where Tglob,30min is the mean of the global transmissivities obtained from the two retrievals just after (before) the SZA fell below (exceeded) 78°. For direct radiation, we assumed that transmissivity is proportional to air mass and thus to the inverse of COSSZA (μ0); hence,
where μ0,30min is the mean of the two COSSZAs of the first (last) two retrievals after (before) the SZA fell below (exceeded) 78°. Apart from missing retrievals for large SZA, a second type of gaps is caused by missing images. These are filled by setting the global and direct transmissivity equal to the mean of the retrieved global and direct transmissivities for the same day.
9.Diffuse transmissivity is calculated as the difference of global and direct transmissivity.
10.Global, direct, or diffuse transmissivity (Ti) are converted into global, direct, and diffuse irradiances (Fi) with the following equation [Deneke et al., 2008]:
where S is the “reduced solar constant” and d is the sun-earth distance in astronomical units. In our calculation, S = 1358.1 W/m2, which is according to Gueymard  the part of the solar constant (1366.1 W/m2) within the wavelength range of the DAK calculations (240–4606 nm). Therefore, SICCS neglects surface solar irradiance outside this wavelength interval.
3 Input Data
 Characteristics of the input data are summarized in Table 2. The following data sets were used:
SEVIRI reflectances to be used for the construction of the cloud mask and as input for the CPP retrieval of the cloud properties. It is the temporal (15 min) and the spatial resolution (3 × 3 km2 at nadir) of these data that sets the resolution of the SICCS output. The same is true for the potential length of the produced time series, namely, from January 2004 until at least September 2012, and the spatial coverage, namely, the part of the Meteosat disc where the satellite view angle is less than 78°. This roughly includes the part of the Earth bounded by Iceland, northern Sweden, Iran, Mauritius, South Georgia, Bolivia, and the Antilles. The spatial resolution degrades toward the edge of the Meteosat disc and is, for example, 3 × 4 km2 in southern Spain, 3 × 6 km2 in Netherlands, and 3 × 9 km2 on the Shetland Islands (east-west and north-south directions, respectively). The cloud mask uses reflectances and brightness temperatures from SEVIRI channels at 0.64, 0.81, 1.64, 3.93, 10.8, and 12.0 µm. The physical cloud properties are derived from observations at 0.64, 1.64, and 10.8 µm. CPP has been developed and is regularly updated by KNMI within the framework of the CM-SAF. The CPP output used as input for the calculations presented in this paper was generated with the latest KNMI version of CPP. KNMI performs near-real time calculations with CPP and provides instantaneous data. Once in a while, the actual version of CPP is delivered to the Deutscher Wetterdienst, where the algorithm is exploited for the production of CM-SAF climate data sets.
AOT at five wavelengths (469, 550, 670, 865, and 1240 nm) modeled by the reanalysis carried out within the framework of the Monitoring Atmospheric Composition and Climate (MACC) project. The MACC calculations are made with the European Centre for Medium-Range Weather Forecasts (ECMWF) Integrated Forecast System and consider five types of aerosol sources, namely, sulfate, black carbon, organic carbon, sea salt, and dust. Moderate resolution imaging spectroradiometer (MODIS) AOT at 550 nm is assimilated into the MACC calculations. For MACC, the Integrated Forecast System is run with a horizontal resolution of approximately 78 × 78 km2, and we downloaded data with a time step of 3 h. Using the validation tool at http://aerocom.met.no/cgi-bin/aerocom/surfobs_annualrs.pl, we compared 4 years of daily values of MACC τa550 with values from Aerosol Robotic Network (AERONET) stations in Europe and the Mediterranean. We found that MACC τa550 is positively biased by 9.3% with respect to the AERONET observations. To correct for this bias, MACC AOT at all five wavelengths was multiplied by a factor of 0.915. Next, using the criterion of least squares, the five values of AOT (τa) at the different wavelengths (λ) were fitted to the following equation:
which yielded values for AOT at 500 nm (τa500) and the AEXP (α). The latter two variables served as input for the radiative transfer calculations.
Monthly climatology of the SSA was produced by the AeroCom project [Kinne et al., 2006]. One of the tasks set by the AeroCom project is to diagnose aerosol modules that form a part of global models. A result of this effort in a global aerosol climatology averaged more than 20 models that we downloaded from http://www.cmsaf.eu. The spatial resolution of the downloaded data is 1° × 1°, but this is an oversampling of the contributing model fields, which typically have a resolution of 3° × 3°. The underlying calculations are monthly averages over a period that varies between models from 1 to 10 years. The downloaded SSA is strictly speaking only valid at 550 nm, whereas our radiative transfer calculations required as input a single, wavelength-independent SSA value. We simply set the input SSA value equal to the downloaded value at 550 nm.
Monthly mean climatological values of IWV for the period 1987–2007 were taken from the ERA-Interim reanalyses. These data have a spatial resolution of 0.25° × 0.25°.
Surface albedos derived from the MODIS surface albedo product MCD43C3. Both CPP and SICCS require surface albedos as input. CPP uses surface albedos at the wavelengths of some of the SEVIRI channels as background for the retrieval of the CPPs. SICCS requires surface albedos in the visible and near-infrared part of the spectrum for the calculation of the extra downwelling radiation due to multiple reflections between the surface and the atmosphere. We downloaded MCD43C3 time series for the white-sky albedos in the visible (0.3–0.7 µm) and near-infrared (0.7–5.0 µm) part of the spectrum and at four wavelengths (659, 858, 1640, and 2130 nm) corresponding to SEVIRI channels at similar wavelengths. Each element of MCD43C3 represents a period of 16 days, but the elements overlap in time resulting in a temporal resolution of MCD43C3 of 8 days. A limitation of this product is that many data points are missing or of inadequate quality. As SICCS requires continuous input fields, we developed a procedure for filling the missing and bad-quality data by inter and extrapolation. This method is described in the appendix. The resulting data fields have the same temporal (8 days) and spatial resolution (0.05° × 0.05°) as the original MODIS product and cover the period from February 2000 to January 2011. It is important to note that unlike the climatological product used in SICCS version 1, we did not filter out the effect of snow and ice on the albedo. Also, unlike the product used in SICCS version 1, the new albedo time series contains interannual variation. Another relevant issue is that although MCD43C3 and our product are available for the continental shelves, they do not exist for the deeper parts of the oceans. We assumed visible and near-infrared surface albedos of 0.05 for those parts of the ocean.
Ozone and its vertical distribution as well as the vertical distribution of the water vapour were taken from the midlatitude summer atmospheric profile of Anderson et al. . The CO2 concentration was set equal to 381 ppm.
Table 2. Main Characteristics of the Most Important Input Data Sets Used in SICCS.
Reflectances at 0.64, 0.81, 1.64, 3.93, 10.8, and 12.0 µm
3 × 3 km2 (nadir)
Aerosol optical thickness at 469, 550, 670, 865, and 1240 nm
0.5° × 0.5°
Single scattering albedo at 550 nm
1° × 1°
Derivate of MODIS MCD43C3
White sky surface albedo at 659, 858, 1640, and 2130 nm, plus visible and near-infrared
0.05° × 0.05°
ECMWF ERA Interim
Integrated Water Vapor
0.25° × 0.25°
 As the processing was carried out for the elements of the grid of SEVIRI pixels, all of the non-SEVIRI input data sets were resampled onto that grid, using the nearest neighbor method.
4 Validation Data
 The product discussed in this paper was validated with radiation measurements from BSRN stations. To assess the effect of errors in the input of the modeled AOT (MACC) on the computed irradiances, we also performed calculations with ground-based aerosol data from AERONET sites. In this section, we will briefly describe the exploited BSRN and AERONET measurements.
 We exploited data from stations of the BSRN network [Ohmura et al., 1998] because BSRN stations deliver their measurements according to the highest available standards. Global, direct, and diffuse irradiance are each measured with independent devices. Shi and Long  estimated the operational uncertainties for BSRN-type measurements to be typically 14 ± 6 W/m2 for direct irradiance and 9 ± 3 W/m2 for diffuse irradiance. For the Atmospheric Radiation Measurement facility, where instruments similar to those at the BSRN sites are operated, Stoffel  gave the following estimates of 2-sigma uncertainties for direct, diffuse, and global irradiance: 3% or 4 W/m2 (whichever is larger), 6% or 20 W/m2, and 6% or 10 W/m2, respectively. The choice of the BSRN stations that we actually selected for the validation was set by the domain (Europe and the Mediterranean region) and the period (the year 2006) of the data that we processed so far. This delimited the number of stations to nine (see Figure 1). One-minute values of global, direct, and diffuse irradiance for the year 2006 and the nine stations were downloaded from the BSRN central archive at http://www.bsrn.awi.de/ and averaged to hourly means to be used for validation. For each of the nine stations, we checked the quality of the measurements by calculating the residual of global irradiance minus the sum of direct and diffuse irradiance. The distribution of the residuals was, among others, characterized by its standard deviation. The lowest values were found for Cabauw, Carpentras, and Lindenberg (5 W/m2) and the highest values for Payerne (38 W/m2) and Sede Boqer (58 W/m2). Also, for Sede Boqer absolute values of the mean and the skewness of the distribution surpassed the values of the other sites. Sede Boqer was therefore excluded from the BSRN validation data set. In the analyses, we also omitted measurements from the other sites if absolute values of the residual were larger than 10 W/m2. The percentage of such low-quality samples varied from 3% for Cabauw to 47% for Payerne.
 In 2006, four of the eight BSRN stations used for our validation studies harbored an AERONET [Holben et al., 1998] station as well, namely, Cabauw, Carpentras, Palaiseau, and Toravere. For these stations, we downloaded level 2.0 data of AOT, SSA, and IWV from http://aeronet.gsfc.nasa.gov/index.html. The sampling rate of these data is irregular up to a maximum of 18 samples per hour (for the four mentioned stations in 2006). We aggregated the instantaneous values to hourly means. The aerosol properties can only be determined for clear skies, but the data were not available for all of the clear-sky hours. We checked this by determining the presence/absence of clouds with the combined information from the SEVIRI cloud mask and the BSRN measurements (see section 6.2). AOT and IWV were available for a relatively large part of the clear-sky hours (62% for Cabauw, 88% for Carpentras, and 91% for Palaiseau and Toravere), but SSA was determined for only 18–55 h of the entire year, depending on the site. AERONET AOT is provided at various wavelengths between 340 and 1640 nm, with on average for the downloaded data values at six different wavelengths per time slot. Using equation (5), these were converted to the AOT at 500 nm (τa500) and the AEXP (α), which served as input for the radiative transfer calculations.
5 Some Important Aspects of the Algorithm
5.1 The Radiative Transfer Model
 The radiative transfer calculations were carried out with the DAK model [de Haan et al., 1987; Stammes, 2001], which was adapted for making broadband calculations by Kuipers Munneke et al.  using the correlated k-distribution technique to account for atmospheric gas absorption. In the broadband version of DAK, the solar spectrum is divided into 32 bands within the wavelength range from 240 to 4606 nm. For this study, we subdivided the atmosphere into the 32 layers given by the input midlatitude summer atmospheric profile of Anderson et al. . Given a layer thickness of 1 km for the lowest layers, the number of layers amounted to 31 and 30 layers for surface elevations of 1 and 2 km, respectively.
 Clouds were assumed to occupy a single layer at a height of 1–2 km above the surface. To compute the effect of clouds on radiative transfer, DAK requires COT, SSA, and scattering phase function as well as their spectral variations as input. However, the available SEVIRI cloud properties consisted of COT (at 640 nm), effective particle radius, and phase. For water clouds, the necessary conversions were performed with Mie scattering calculations. For ice clouds, we made the conversion with the ray-tracing program SPEX [Hess et al., 1998], assuming that the ice crystals have a hexagonal shape and an irregular surface. The Mie scattering and ray-tracing calculations are analogous to similar calculations made for the retrieval of the cloud properties with CPP.
 Aerosol is assumed to occupy the lowermost atmospheric layer. Again, like for clouds, DAK requires AOT, SSA and scattering phase function as well as their spectral variations as input. However, the available input consisted of AOT at 500 nm, the AEXP (α), and a single value of the SSA. The spectral variation of AOT is directly given by α. The scattering phase function is approximated by a Henyey-Greenstein phase function with the asymmetry parameter (g) given by:
as based on Mie simulations (Stefan Kinne, personal communication). The SSA and the Henyey-Greenstein function are assumed to be constant with wavelength. Although both do vary with wavelength, the assumption of constancy has a negligible effect on the calculations [Wang et al., 2009].
 The broadband version of DAK was tested in several studies. Wang et al.  made a closure study for 72 clear-sky cases in Cabauw. They found mean differences between calculations and measurements of +2, +2, and +1 W/m2 (all less than 1%) for global, direct, and diffuse irradiance and 3, 3, and 2 W/m2 for the root mean square errors (RMSEs) of the same irradiances. This study was then repeated for 639 selected cases of fairly homogeneous single-layer water clouds at the same location [Wang et al., 2011a], using cloud liquid water path from surface microwave radiometer data and droplet radii from MODIS as input. Relative bias and RMSE of global irradiance (5% and 13%, respectively) were much larger than in the clear-sky cases, but absolute bias and RMSE were still relatively small (+6 and 14 Wm−2, respectively). We also performed DAK calculations for the seven cases selected by Continual Intercomparison of Radiation Codes [see Oreopoulos and Mlawer, 2010]. All of the computed global and diffuse transmissivities agreed with the observations and the line-by-line calculations provided by Continual Intercomparison of Radiation Codes within 0.008. We conclude that the accuracy of DAK for clear skies and homogeneous water clouds is excellent. DAK has not been validated for ice clouds.
5.2 Direct Radiation for Cloudy Atmospheres
 As discussed in section 2, we estimate global transmissivity for cloudy atmospheres by performing forward radiative transfer calculations, prescribing the clouds by the properties retrieved with CPP and assuming that the clouds are plane parallel. There are good reasons to believe that this approach fails for direct radiation in the case of cloudy atmospheres. In calculations, direct radiation is quickly attenuated by plane-parallel clouds; whereas in reality, direct radiation is, to a first approximation, determined by its clear-sky value and the fraction of time that the surface is exposed directly to the sun. In other words, the fraction of the sky occupied by gaps in the cloud field is a crucial factor. However, subpixel size gaps are, of course, not seen by satellite sensors, so their effect cannot be quantified from satellite data with radiative transfer calculations.
 We tested the hypothesis that direct irradiance under cloudy skies cannot be calculated by plane-parallel calculations by comparing such calculations with the data (Figure 2). The dashed line in the figure represents the fraction of monochromatic radiation at 640 nm that is transmitted by a plane-parallel clouds of a given COT (in CPP 640 nm is the wavelength for which COT is retrieved). Because clouds absorb more at other wavelengths, this relation constitutes an overestimation of the equivalent broadband relation. Measured cloud transmittance is computed by exploiting the BSRN measurements of direct transmissivity (Tdir,meas) in combination with modeled estimates of direct transmissivity for a clear sky (Tdir,clr), where Tdir,clr was computed with the clear-sky direct irradiance LUTs and aerosol input from AERONET stations. This approach confined data sources to combined BSRN/AERONET sites (Figure 1). We collected data from Carpentras, Palaiseau, and Toravere, where each sample (n = 11686) consists of a COT retrieved from a single image and for the pixel containing the station, and a 15 min mean of Tdir,meas/Tdir,clr. The samples were then separated into a collection for water clouds and a collection for ice clouds. Next, for each type of cloud, the samples were collected into bins of sorted COT values with 400 samples per bin, and bin-averaged values of Tdir,meas/Tdir,clr were plotted against COT (the triangles in Figure 2). Obviously, in a statistical sense, measured cloud transmittance is substantially larger than transmittance for plane-parallel clouds. Therefore, plane-parallel calculations of the transmittance of direct radiation through clouds constitute a severe underestimation and the data are in agreement with our “gap hypothesis.”
 To consider this effect in the algorithm, we made least squares fits of equation (1), one for water clouds and one for ice clouds, to the bin-averaged measurements of Tdir,meas/Tdir,clr, resulting in the values for the constants given in section 2 and the solid lines depicted in Figure 2. There are still two interesting issues concerning Figure 2. First, there is a distinct difference between the behavior of water and ice clouds. For the same COT, water clouds attenuate direct radiation much less than ice clouds do. For plane-parallel clouds, this is unexpected because ice clouds have a much stronger forward scattering peak than water clouds. Because a small part of the forward scattered radiation is interpreted as direct radiation by pyrheliometers, the peak in the angular distribution of ice crystals would lead to a larger measured transparency of direct radiation for ice clouds than for water clouds. However, the measurements show the opposite behavior, which according to the gap hypothesis suggests that at subpixel scale water clouds are more broken than ice clouds, on average. Second, we like to note that exp(−A) (= 0.035 for water clouds and 0.019 for ice clouds) is the fraction of the clear-sky direct surface irradiance that still reaches the surface in the presence of the thickest clouds, in a statistical sense.
5.3 Differences Between SICCS Version 1 and Version 2
 The most important modifications in SICCS version 2 with respect to version 1 are as follows:
DAK replaces discrete ordinates radiative transfer so that the inverse radiative transfer calculations of CPP and the forward radiative transfer calculations with SICCS are carried out with the same model.
Direct and diffuse irradiance are computed, which was not performed in version 1.
A gap-filling procedure for direct surface irradiance is added.
For clear skies, spatial and temporal variations in aerosol properties and spatial variations in surface elevation are considered. In version 1, aerosol load and type were constant across the entire domain and for all time slots.
The new surface albedo data set includes both the effect of snow on the albedo and the interannual variations of the albedo. In version 1, a snow-free albedo climatology was used as input.
The IWV input generated by the CM-SAF is replaced by ECMWF climatology.
Calculations are extended from a limiting SZA of 72° to a maximum angle of 78°.
The solar constant is adapted to the wavelength range of DAK, which was not performed in version 1.
 We processed data for the entire year of 2006 and validated the result by comparison with ground-based BSRN measurements. For all of the validations, we used hourly mean data (four satellite images and 60 BSRN measurements), which complies with other validation studies [e.g., Deneke et al., 2008; Huang et al., 2011]. To compute the hourly mean satellite values, retrievals are performed for each satellite image, and the resulting irradiances (four per hour) are averaged. Unless otherwise mentioned, we compared the BSRN measurements with the retrieval results from the pixel containing the BSRN site. To distinguish the contribution of different factors to the overall performance of the algorithm, the validation is performed in several steps: we start with clear-sky cases, for which we investigate the performance of the standard calculations as well as the effect of using locally measured instead of modeled aerosol input and the effect of temporal resolution of the input data. We then step to atmospheres with water clouds and atmosphere with ice clouds, after which we take all cases together and make an analysis. Finally, we validate cases with snow at the surface and cases of large SZA and investigate the sensitivity of the retrieval biases to cloud and other parameters. Validation statistics for all eight BSRN stations are summarized in Table 3 whereas most figures are restricted to the stations of Cabauw (because this station scored best in terms of the quality of the BSRN measurements) and Carpentras (because this is the only station in the Mediterranean region). Scatterplots of observed against retrieved samples of irradiance and transmissivity are mainly evaluated in terms of the (absolute) bias, the relative bias, the (absolute) RMSE, and the relative RMSE.
Table 3. Statistics of the Validation of Hourly Mean SICCS Retrievals With BSRN Measurementsa
The four groups of columns give the absolute biases, the relative biases, and the median values of the biases and RMSEs for all the BSRN sites. The BSRN stations are located in Cabauw (cab), Camborne (cam), Carpentras (car), Lerwick (ler), Lindenberg (lin), Palaiseau (pal), Payerne (pay), and Toravere (tor). The seven groups of rows refer to different validation experiments. Individual rows contain results for direct, diffuse, and global irradiance.
6.2 Validation for Clear Skies
 A good motivation for separately validating clear-sky cases is that in SICCS estimation of the irradiance for clear-sky cases hardly depends on input of satellite data, except for a small effect of the MODIS surface albedo. Hence, for clear skies, the performance of the algorithm mainly depends on the quality of the other input data sets, especially those of aerosol properties and of IWV, and the performance of the radiative transfer model.
 An hourly mean combination of satellite and BSRN data was designated as a clear-sky case when all of the following criterions were fulfilled:
On all four images, the satellite pixel nearest to the station was clear according to the cloud mask.
The standard deviation of the global transmissivity of the BSRN samples was less than 0.0025.
The ratio of measured direct to measured global irradiance was greater than 0.4.
 We started the validation with an idealized setup, in which aerosol optical properties and IWV were given by local AERONET measurements. This restricted the SICCS calculations to those hours for which AERONET AOT and IWV data are available (see section 3) and to the four sites that harbor both a BSRN and an AERONET station. As mentioned in section 3, in 2006, SSA data from the AERONET stations were sparse. SSA values for hours without SSA data were computed from the available SSA data by linear interpolation in time.
 Results are shown in Table 3 and Figure 3, where, like in all other BSRN validation plots, different symbol colors represent the four seasons and the dashed line shows the best linear fit through the data using the criterion of least squares in the direction perpendicular to that of the fitted line. The bias is defined as the difference between the mean of the SICCS calculations and the mean of the BSRN measurements. Absolute biases are small for all stations (median values of 5, 4, and 9 W/m2 for direct, diffuse, and global irradiance, respectively). In terms of relative biases this corresponds to 1%, 4%, and 2%, respectively. Note at this point that the sum of the biases in direct and diffuse irradiance must not necessarily be equal to the bias in global irradiance because this equality does not hold for the BSRN measurements. Overall, there is a tendency toward positive biases for all surface irradiances and stations, but biases are all smaller than the uncertainties in the measurements discussed in section 4. RMSEs are also relatively small, with median values of 9, 10, and 14 W/m2 for direct, diffuse, and global irradiance (2%, 10%, and 3%, respectively). Although our results are, in view of the uncertainty in the BSRN measurements, satisfying, we note that Wang et al. , also using local AERONET data as input, obtained much better closure between BSRN measurements and DAK calculations (see section 5.1). They possibly achieved part of their greater accuracy by selecting instantaneous values instead of hourly means. Moreover, Wang et al.  disposed of AERONET SSA data for the investigated days, whereas the AERONET SSA input for SICCS was based on a very limited number of measurements (18–55 for the entire year) and interpolation. Moreover, all samples selected by Wang et al.  were obtained under skies without cirrus according to human observations, whereas part of our samples could be contaminated by cirrus that was not detected by the cloud mask algorithm. To test to some extent the effect of the SSA input on our results, we replaced the interpolated AERONET input by the monthly mean SSA input from the AeroCom project, which is the standard SSA input of SICCS. Because the SSA has no influence on direct radiation, the change in input had no effect on direct irradiance. The biases in diffuse and global irradiance increased, depending on the station, by between 2 and 10 W/m2, which provides an indication of the sensitivity of the calculations to uncertainties in the SSA input.
 SICCS requires input that is continuous in time and space. Of course, this requirement is not fulfilled by the AERONET measurements. Also, satellite observations of clear-sky aerosol properties and IWV are generally not temporally complete. We therefore exploited AOT from the MACC project, SSA from the AeroCom project, and IWV from ECMWF reanalyses as the standard input for the clear-sky SICCS calculations. This standard setting was tested in a further validation experiment, of which some results are presented in Figure 4 and Table 3 (“clear sky, MACC input”).
 Compared with validation with AERONET input, biases at the combined BSRN/AERONET sites shift slightly for global irradiance (depending on the site, by between −13 and +8 W/m2, which is by between −2% and +2%). In fact, agreement of computed global irradiance with the BSRN observations improved at all four stations. Regarding direct irradiance, relatively large negative biases (between −12 and −32 W/m2, which is between −2% and −6%) occur at five of the eight stations. These negative biases in direct irradiance are all paired with relatively large positive biases in diffuse irradiance (between +8 and +27 W/m2, i.e., 13% and 32%), whereas at four of the five sites, the bias in global irradiance is relative small (−2% to 0%). A potential explanation for the changes in the validation results due to replacing AERONET by MACC input could have come from a difference in the selection of samples. We checked this possibility by running SICCS with MACC input, but only for hours when AERONET input was available. Results were almost identical to the calculations for all clear-sky cases. Hence, the changes in the validation metrics, when AERONET is replaced by MACC input, are not due to data selection but must be ascribed to shifts in the input variables. Also, we remark that the quality of the MACC input variables is less than the quality of the AERONET data. Therefore, the better agreement of global irradiance with the observations must be ascribed to compensation between biases in the ground measurements, in DAK calculations, and/or in the input variables. The pairing of negative biases in direct irradiance with positive biases in diffuse irradiance suggests that the input AOT is too large at the pertinent stations. Note that the pairing of the biases cannot be ascribed to errors in IWV and SSA input because a bias in IWV would change the biases in direct and diffuse irradiance in the same direction and a bias in SSA would only affect diffuse irradiance.
 Substitution of the AERONET input by the standard SICCS input has a small impact on the RMSEs of global irradiance, but the RMSEs of direct and diffuse irradiance increase substantially (by factors of 4 and 3, respectively). These findings suggest, not surprisingly, that for clear-sky conditions, the temporal variations in the standard input match the real variations to a much smaller extent than the AERONET input does. Errors in AOT input have more effect on direct and diffuse irradiance than on global irradiance, whereas errors in IWV have an effect of similar magnitude on all irradiances [Wang et al., 2009]. We therefore conclude that the negative effect of the full model input mainly resides in the AOT part.
 Many surface solar radiation algorithms use monthly or longer term means of aerosol optical properties and IWV as input. An example is the previous version of SICCS, which assumed constancy in time and space of all aerosol properties. To compute the sensitivity of taking monthly averages as input, we averaged the three-hourly MACC aerosol input (AOT500 and the AEXP) to monthly means and validated the clear-sky cases again. Results are shown in Figure 5 and Table 3. It is not surprising that the lower temporal resolution of the input leads to larger RMSEs, but there is also a noteworthy effect on the biases. Biases in direct irradiance shift significantly downward (the median by −33 W/m2 or −6%), whereas biases in diffuse irradiance shift upward by almost similar amounts (the median by +27 W/m2 or +26%). The effect of averaging was much smaller on the biases in global irradiance, which changed by −8 W/m2 (−1%). We repeated the calculations with daily means of the MACC aerosol input and found shifts with respect to the calculations with three-hourly input of −11 W/m2 or −2% in direct irradiance, +7 W/m2 or +7% in diffuse irradiance and −4 W/m2 or 0% in global irradiance. So the bias introduced by averaging AOT increases with increasing length of the averaging interval. We checked whether the biases were possibly caused by conditional sampling in the standard calculations with MACC input (“clear-sky MACC input”), namely, sampling for SZAs less than 78° and clear-sky conditions. It appeared that in the MACC data set and on average over the validation sites, annual mean AOT was almost equal to (2% smaller than) the mean AOT for SZAs less than 78° and clear-sky conditions. We conclude that conditional sampling does not explain the biases due to averaging the input but that the biases are caused by the nonlinear relationship between AOT and irradiance. Figure 6 illustrates how performing calculations with averaged AOT introduces biases and gives an indication of the sign and the relative magnitude of the biases for the three irradiances. The effects of averaging demonstrated in the figure correspond qualitatively to what we found in our analyses with the MACC aerosol input data set.
 Because of the relationship between the IWV and the irradiances, which is also not linear, monthly IWV input, as used in SICCS, will introduce biases similar to those due to averaging the aerosol properties. We could not quantify the effects in the same way as we investigated the effect of averaging the aerosol properties because we did not have ECMWF analyses at high temporal resolution at hand. Instead, we tested the effect of averaging IWV with the AERONET data. We computed monthly mean values of AOT, AEXP, and IWV from the AERONET data and carried out two sensitivity experiments: one with monthly mean AOT and AEXP and hourly IWV, and one with monthly mean IWV and hourly AOT and AEXP. Both experiments introduced biases with respect to calculations with hourly input of AOT, AEXP, and IWV, but it appeared that for all three irradiances, the biases caused by averaging IWV were approximately one eighth of the biases caused by averaging AOT and AEXP. Hence, averaging IWV has a much smaller effect on mean values of the irradiances than averaging AOT and AEXP. We assume that this conclusion is not only valid for AERONET input but also for other input data sets of AOT, AEXP, and IWV.
6.3 Validation for Cloudy Skies
 On the subpixel scale, clouds generally deviate much more from the assumption of horizontal homogeneity underlying the SICCS radiative transfer calculations than other atmospheric constituents like aerosol and water vapor. The inhomogeneity may cause substantial errors in estimating cloud properties from satellite data [e.g., Marshak et al., 2006; Zinner and Mayer, 2006; Zhang and Platnick, 2011]. Hence, clouds are expected to have a negative effect on the performance of the irradiance retrievals. Also, horizontal inhomogeneity combined with mismatches in sensor footprint and in temporal resolution between ground-based and satellite data causes errors due to the validation itself [Deneke et al., 2009; Greuell and Roebeling, 2009]. Such errors are expected to be random. Not surprisingly, in our validation for water cloud, RMSEs have median values of 80 W/m2 for direct, 65 W/m2 for diffuse, and 75 W/m2 for global irradiance, thereby exceeding the values for clear skies (37, 28, and 15 W/m2, respectively) by factors of approximately 2.2, 2.3, and 5.0, respectively (Figure 7 and Table 3). When compared with water clouds, ice clouds have a somewhat smaller absolute RMSE of global irradiance (61 W/m2), which is at least partly because ice clouds tend to be thicker than water clouds (median values of station-averaged global transmissivities for ice clouds and water clouds are 0.27 and 0.39, respectively). RMSEs for direct irradiance under ice clouds are much smaller (27 W/m2) than under water clouds (80 W/m2), mainly because the direct transmissivity is much smaller for ice clouds (0.04) than for water clouds (0.12). There are two reasons why ice clouds transmit only such small amounts of direct radiation. First, as mentioned before, they are on average optically thicker than water clouds; and second, for ice clouds, the decay of direct transmissivity with COT is faster than for water clouds (see Figure 2).
 In the scatterplots for direct irradiance shown in Figure 7, the linear fit to the samples (dashed line) has a derivative much smaller than 1.0 (0.693 for water clouds and 0.500 for ice clouds). This can largely be ascribed to two choices made to produce all of our scatterplots, namely, to show irradiance and to show the linear fit with minimum total squared distance in the direction perpendicular to the line itself. However, equation (1) was derived by applying the least squares criterion to transmissivity in the BSRN measurements. If in Figure 7 irradiance is replaced by transmissivity and the sum of the squared deviations in the x-direction is minimized, the derivatives are much closer to 1.0 (0.952 for water clouds and 0.874 for ice clouds), which demonstrates that the equation, which was derived from BSRN measurements made at Carpentras, Palaiseau, and Toravere, applies fairly well to Cabauw.
 Both at the level of individual sites and at the level of median values for all sites, absolute biases for clouds (“Water Clouds” and “Ice Clouds” in Table 3) tend to be equal or smaller than the biases for clear skies (“Clear-sky MACC Input” in Table 3). The same is true even for the relative biases except large relative biases of direct irradiance for ice clouds. In summary, clouds have a negative effect on the precision (RMSE) of the retrieval but they do not lead to a loss of accuracy (biases). Lower precision is partly caused by errors due to the validation itself.
 We need to mention here that in an initial validation, significant positive biases in global transmissivity for ice clouds were found at all eight sites (between 0.028 and 0.070; between and 18 and 47 W/m2 for global irradiance). For water clouds, we also found systematic but smaller positive biases in global transmissivity (0.011 on average). The fact that the biases for water clouds are much smaller than for ice clouds points toward a flaw in the single scattering properties of the ice crystals. With the aim of bringing the calculated irradiances closer to the ground-based measurements, we introduced a correction for the ice-cloud bias, namely, subtraction of a constant (0.0375, which is the median of the biases of the eight stations) from the global transmissivity. As a result, the bias for ice clouds when averaged over the sites is almost equal to zero. The global transmissivity for water clouds was not corrected.
 We would finally like to note that many of the largest positive biases for cloudy skies were found at Lerwick and Toravere. Among the considered BSRN stations, these two stations have the largest SEVIRI viewing angles (68° and 71°, respectively), which suggests that biases in the estimation of solar irradiance for cloudy atmospheres grow toward the edge of the images. Also, water clouds at Lerwick are probably, in a statistical sense, more homogeneous than the clouds at the three sites used to develop the equation for direct transmissivity under cloudy skies (equation (1)). This means that at Lerwick direct irradiance is closer to values expected for plane-parallel clouds, given by the dashed line in Figure 2, than to the values given by equation (1) and the red curve in Figure 2. Thus, at Lerwick characteristics of the local water clouds also likely contribute to the large biases of direct (+30 W/m2) and diffuse (−15 W/m2) irradiance for this type of clouds.
6.4 Validation for All Cases
 The most crucial validation test is a validation including all cases (“all sky”). Figure 8 shows results for Cabauw and Carpentras. Taking all of the eight validation sites, Table 3 shows that biases are slightly positive for direct irradiance, namely, between 0 and +12 W/m2 (0% and +6%), except for Lerwick where the bias amounts to +30 W/m2 (+75%). Retrieved diffuse irradiance is, on average over the stations, almost equal to measured irradiance, with biases between −4 and +4 W/m2 (−2.3% and +2.3%), but biases are larger for Lerwick (−15 W/m2; −9% ) and Toravere (+17 W/m2; +14%). Table 3 also shows that when direct and diffuse irradiance are combined into global irradiance, the bias is generally slightly positive with values between −1 and +7 W/m2 (−0.2% and +1.9%). Relatively, large biases are found for Lerwick (+12 W/m2, +5.8%), Payerne (+12 W/m2, +3.3%), and Toravere (+13 W/m2, +4.0%).
 For direct irradiance, the absolute value of the relative bias exceeds the 1-sigma uncertainty given by Stoffel  (1.5%) at six of the eight BSRN stations, so many of the positive biases in retrieved direct irradiance are significant. The source of the deviations can be either in the input data, e.g., an underestimate of AOT, or in the radiative transfer calculations but the deviations can, of course, also be due to a combination of these two sources of error. For diffuse and global irradiance, the biases are within the uncertainties of the measurements given by Stoffel , i.e., 3%, with the exceptions of the stations of Lerwick and Toravere as well as Payerne as far as global irradiance is concerned. As mentioned before, the deviations for Lerwick and Toravere can be attributed to the deviations for cloud cases occurring under unfavorable view zenith angles. We also tried to track down the source of the large relative bias in global irradiance at Payerne (+3.3%). Possibly, a part of this deviation is caused by an underestimate of IWV by the ECMWF analyses at this somewhat elevated (491 m a.s.l.) site.
 The positive bias in global irradiance is mainly due to an overestimate for water clouds. For direct and diffuse irradiance, compensating biases for the diverse types of sky (clear, water clouds and ice clouds) contribute to the relatively small overall biases. SICCS overestimates direct irradiance for cloudy atmospheres but underestimates this component of radiation for clear skies. Diffuse irradiance is underestimated for ice clouds and overestimated for clear skies.
6.5 Sensitivities: Snow and Large SZAs
 Snow-covered surfaces and large SZAs are two conditions under which the performance of the algorithm might degrade. Here, we present validations for these two types of conditions. According to Deneke et al. , the presence of snow-covered surfaces significantly degraded the accuracy of the first version of the SICCS retrieval. In fact, snow was not taken into account because the albedo background maps were valid for snow-free surfaces. As discussed earlier on, we replaced the background maps by data sets that include the effect of snow. However, even with such improved background albedo maps, snow can be expected to degrade the skill of solar irradiance algorithms. Snow decreases the capacity of the cloud mask to detect clouds and hampers the retrieval of cloud properties because clouds and snow tend to have similar effects on radiative transfer and therefore on reflected radiation at the TOA.
 We applied SICCS in a straightforward way to snow pixels, which means that snow pixels are dealt with in exactly the same way as other pixels. For the validation, snow-covered surfaces were selected by setting a threshold of 0.4 to the MODIS albedo in the visible part of the spectrum. Results are presented in Figure 9 in terms of global transmissivity and in Table 3 in terms of global irradiances. Our assessment is limited to Lindenberg and Toravere because at the other stations the visible albedo never exceeded the threshold of 0.4 in 2006. Obviously, the correlation coefficients (r) for transmissivity are highly significant. At Toravere r (0.918) almost matches the r for all cases (0.925), and at Lindenberg r is even larger (0.960) than its value (0.886) for all cases. This unexpected result at Lindenberg is perhaps due to the limited amount of snow data at this site (47 h) and the fact that most of the selected snow data were collected when the atmosphere was either clear or covered by thick clouds, leading to the bimodal distribution of the samples. For snow-covered surfaces, absolute biases and RMSEs of global irradiance (Table 3) are smaller than their values for the validation of all cases, but these statistics are favored by the relatively small incoming radiation at the TOA when surfaces were snow-covered in Lindenberg and Toravere. Nevertheless, even in a relative sense, the snow retrievals of global irradiance do not appear to be inferior to retrievals for other conditions. Although the relative bias was +1% at Lindenberg and +4% at Toravere, the snow values are −2% and −1%. Relative RMSEs of global irradiance were 21% (Lindenberg) and 18% (Toravere) for all cases, and 17% at both sites for snow cases.
 In SICCS version 1, the algorithm was only applied to pixels with an SZA less than 72° because this is the maximum angle for which the standard CPP product is available. This limit was set in CPP because uncertainties in cloud properties become unacceptably large at larger SZA. In SICCS version 2, we continued processing with both CPP and SICCS up to an SZA limit of 78°. Figure 10 shows scatterplots of irradiance and transmissivity at Cabauw for SZA between 72° and 78°, and Table 3 shows some irradiance statistics for the other sites. Biases are generally very small with absolute values smaller 3 W/m2 at five of the seven stations (in Carpentras the sun rose and set more quickly than at the other stations; hence, during none of the hours, it remained entirely within the requested SZA range). The station-median absolute RMSE (30 W/m2) is much smaller than the value for all cases (65 W/m2) at the same sites. However, at large SZA, incoming solar radiation at the TOA is relatively small, so relative RMSE for large SZA (25%) is somewhat larger than the value for all cases (18%). If judged by the correlation coefficient, the skill of retrieving global irradiance diminishes from 0.945 for all cases to 0.901 for large SZA cases. The latter value is still highly significant, which justifies the extension of the calculations up to an SZA of 78°.
6.6 Sensitivities to Cloud and Other Parameters
 We also investigated whether the biases varied systematically with cloud properties, cloud homogeneity, and SZA. For this purpose, we collected the hourly data for all of the eight BSRN sites and plotted the residuals of the calculations and the measurements against COT, cloud particle radius, COSSZA, and variability in the BSRN measurements (σBSRN). The latter variable can be considered as a measure of cloud homogeneity on pixel and subpixel scale. This analysis was carried out for both global and direct irradiance, for three types of cloud conditions (clear sky, water clouds, and ice clouds), and to eliminate the trivial dependence of the irradiances on the SZA in terms of transmissivity. The most important relationships, all significant at least at the level of 99.9%, that were found are shown in Figure 11. For both water and ice clouds, the bias in global transmissivity decreases systematically with increasing variability in the BSRN measurements as shown by the line representing the best linear fit to the bin-averaged values. For homogeneous cloud conditions (σBSRN = 0), the bias is positive (+0.024 for water clouds and +0.040 for ice clouds). Thus, model calculations are positively biased when clouds fulfill the assumption of plane-parallel clouds. Note that for ice clouds, the bias would have been larger by +0.0375 without the empirically determined correction that we applied to the transmissivity of these clouds. For extremely inhomogeneous conditions (σBSRN = 0.3), the bias is negative (−0.030 for water clouds and −0.140 for ice clouds). Hence, cloud inhomogeneity contributes toward negative biases in the global irradiance calculations. The sensitivity of transmissivity to the amount of heterogeneity, given by the derivative in the figures, is larger for ice clouds (−0.60) than for water clouds (−0.18).
 Another interesting relationship that we found is shown in the lower panels of Figure 11. The bias in direct transmissivity is a function of COSSZA in the sense that it is positive when the sun is low (e.g., +0.048 for water clouds and +0.026 for ice clouds if COSSZA = 0.3) and negative when the sun is high above the horizon (e.g., −0.027 for water clouds and −0.007 for ice clouds if COSSZA = 0.8). In a statistical sense, the relation between the bias in direct transmissivity and COSSZA can well be described by a linear relationship as illustrated by the best fit to the bin-averaged values. This relationship can be explained by the probability that solar rays reach the surface directly through gaps in a cloud field. Provided neither gap pattern nor cloud geometric thickness varies with the SZA, the probability that a solar ray reaches the surface directly through a gap without being scattered by the clouds decreases with the SZA. Therefore, direct transmissivity decreases with increasing SZA, an effect that is not accounted for by equation (1).
 As mentioned earlier, we looked at many other relationships between irradiance biases and cloud properties, among others, than those shown in Figure 11. It is beyond the scope of this paper to discuss them all, but it is noteworthy that we found that the biases in global transmissivity were not a function of the SZA, cloud particle radius, or COT, with the exception that for the thickest water and ice clouds (COT > 40), global transmissivity was underestimated by approximately 0.03. We finally like to mention that at none of the BSRN sites the bias exhibited seasonal variation.
 In this paper we, discussed the calculation and the validation of a data set of surface solar incoming radiation. The data consist of global, direct, and diffuse irradiance for the entire Meteosat disc, have a spatial resolution of 3 × 3 km2 at nadir and a temporal resolution of 15 min, and cover the period from 2004 to at least September 2012. As direct and diffuse irradiance are available, the data set is suitable for the calculation of the irradiance on surfaces with any orientation, e.g., solar panels. The main tools used to produce the irradiance data set are the algorithm (SICCS, version 2) and a variety of input data sets, of which the CPP cloud mask product and cloud properties based on Meteosat SEVIRI observations are the most important. Other crucial input data sets are AOT from MACC reanalysis and surface albedo derived from the MODIS MCD43C3 product. CPP/SICCS products are visualized on http://msgcpp.knmi.nl.
 The algorithm is a “physics-based algorithm” as it is based on calculations with a detailed radiative transfer model (DAK), which was also used to calculate the input CPPs. There are two issues, which give an empirical flavor to the algorithm, namely, the subtraction of a constant (0.0375) from global transmissivity for ice clouds and the empirical relation for the estimation of direct transmissivity for cloudy cases. For the derivation of both the constant and the empirical relation, we exploited BSRN surface irradiance measurements.
 We validated hourly means retrieved with SICCS with data from eight BSRN stations for the year 2006 for clear and cloudy skies. SICCS estimates clear-sky irradiances with great skill when AOT and IWV input are taken from local AERONET measurements. Station-median relative biases are +1%, +4%, and +2% for direct, diffuse, and global irradiance, respectively, whereas station-median RMSEs amount to 2%, 10%, and 3% for the same irradiances. Replacing the AERONET input by the standard SICCS input, namely, MACC AOT and ECMWF IWV, relative biases in global irradiance remain small (between −2% and +1% at six stations). However, direct irradiance is mostly negatively biased (between −6% and +1%), and diffuse irradiance is positively biased (between +2% and +32%), which may both be ascribed to an overestimate of MACC AOT. Because the MACC and ECMWF input represent temporal variations in AOT and IWV less precisely than the local AERONET data, station-median relative RMSEs increase to 8%, 29%, and 3% for the three irradiances. We then explored the effect of prescribing the MACC AOT input as monthly means instead of the standard input, which has a temporal resolution of 3 h. As expected, this replacement led to higher relative RMSEs, but the monthly input also caused biases, especially in direct (−9%) and diffuse (+40%) irradiance. These biases are due to the nonlinear relationship between AOT and transmissivity. We conclude that the accuracy of solar irradiance calculations depends critically on the representation of high-frequency variations in the input data sets. As demonstrated this is true for AOT, for which we recommend input containing variations down to the daily timescale, input that includes diurnal variations is even better. The same applies to the cloud properties, which have an even stronger nonlinear relationship with the irradiances. Variability in the remaining input parameters, e.g., IWV, has a much smaller effect than variability in AOT on mean irradiances.
 For cloudy skies, relative RMSEs of global irradiance (30% for water clouds and 31% for ice clouds) are much larger than for clear skies (3%), which is partly due to errors caused by the validation itself (see section 6.3). Relative biases of global irradiance are small for water clouds (+2%) and ice clouds (−1%). For cloudy cases, the performance of the algorithm was found to be inferior for two stations near the edge of the Meteosat disc. On average, over all clear and cloudy cases, SICCS irradiances are somewhat too large, namely, 5% for direct irradiance, 1% for diffuse irradiance, and 2% for global irradiance, whereas RMSEs for the same fluxes amount to 39%, 34% and 18%, respectively. The performance of SICCS is comparable with the performance of SolarGIS, a database of solar irradiances that also uses SEVIRI data as its most important input source. Although SICCS is based on radiative transfer calculations, the SolarGIS algorithm is based on the self-calibrating HELIOSAT method. Šúri et al.  validated SolarGIS output with measurements from 60 stations in Europe and Africa and found a bias of 1.1% and an RMSE of 18.5% for hourly mean global irradiance.
 As suggested in a study about the methodology of validating satellite-retrieved liquid water path with ground-based microwave radiometer measurements by Greuell and Roebeling , the optimum length scale for averaging satellite data is equal to the true image resolution, which is about twice the grid point distance for SEVIRI. Another methodological issue was discussed by Deneke et al. , who compared SEVIRI TOA reflectances with ground-based measurements of atmospheric transmissivity in Netherlands and Germany. The best correlation between the two time series was found when the satellite data were shifted by about 1 pixel (6 km) to the north, which is likely attributable to the parallax effect caused by the height of cloud tops and the oblique satellite viewing angle. As already mentioned, in the standard settings of the present study, we compared the BSRN measurements with the satellite-retrieved irradiances for the pixel containing the BSRN site. To test the suggestions of Greuell and Roebeling  and Deneke et al. , we also compared the measurements of each BSRN station with the means of the irradiances for the 3 × 3 pixels nearest to the site, with the irradiances for the pixel north of the pixel containing the site and with the means of the irradiances for the 3 × 3 pixels around the pixel north of the site. These different validation settings had negligible effects on the biases for clear and cloudy skies and on the RMSEs for clear skies. However, the RMSEs for cloudy skies and therefore also for all skies decreased. Taking the mean of 3 × 3 pixels reduced the station-median all-sky RMSEs for direct, diffuse, and global irradiance (39%, 34%, and 18% with standard validation settings) by absolute values of 2%, 3%, and 0.4%, respectively. Taking the pixel north of the site led to reductions of 3%, 3%, and 1.6% for the same fluxes and if the 3 × 3 pixels around the pixel north of the site were taken, the reductions amounted to 5%, 5%, and 1.8%, respectively. We conclude that short-term variations in the surface irradiances at a single site are better represented by the means of retrievals from 3 × 3 pixels. The favorable effect of the northerly shift confirms the findings of Deneke et al. , but note that the parallax effect depends on the satellite view and azimuth angle and will hence not be uniform across the Meteosat disc.
 Major shortcomings of the first version of SICCS, like no calculation of direct and diffuse irradiance, no consideration of temporal and spatial variations in the aerosol content of the atmosphere, and a background albedo map that was snow-free and lacked interannual variation, have been overcome by the new version. Nevertheless, shortcomings and limitations remain. Without the correction that we made, ice-cloud global transmissivities were on average too high (by 0.0375). This overestimate may suggest that absorption in ice crystals is underestimated by the radiative transfer calculations (see hereafter). As a rather trivial drawback, the restriction of SICCS to the Meteosat disc could be mentioned. Also, at present, SICCS is unable to take into account the effects of relief, such as horizon shading and reflections from surrounding terrain [Dürr and Zelenka, 2009; Lee et al., 2011]. In principle, relief could in future be accounted for by a postprocessing algorithm. SICCS considers the effect of elevation on irradiance, but this capacity was not tested. The reason is that the range of surface elevations of the BSRN sites used for the validation (0 – 491 m a.s.l.) is too narrow for such tests.
 It is of interest to briefly discuss why algorithms used to estimate surface solar irradiance from satellite data like SICCS perform fairly well, what limits their skill, and why cloud inhomogeneity has a relatively mild effect on the accuracy of the calculations (upper panels of Figure 11). Basically, the favorable results of retrievals of global irradiance from satellite data are a consequence of energy conservation. At the TOA, the incoming solar radiation is known with great accuracy, whereas the outgoing solar radiation is measured by the satellite. Then, to a first-order approximation and assuming energy conservation, the solar radiation reaching the surface equals the difference between these two irradiances. This, of course, is an oversimplified picture. For precise calculations, one needs to take into account that there is absorption in the atmosphere, that the surface reflects a part of the incoming irradiance, and that the satellite instruments measure the radiation from a single direction only and in a few narrow bands only. All of these complicating factors are, in principle, taken in account by the radiative transfer calculations, so it is the accuracy of the calculations of the absorption and the directional distribution of the outgoing radiation, among others, that determine the accuracy of the computed surface solar irradiance. Indeed, the accuracy of the satellite measurements of the reflected narrowband radiances should be added to this list. The favorable results of the validation of global irradiance in this and other studies [e.g., Dürr et al., 2010; Journée et al., 2012] demonstrate that the errors made in the calculations of absorption and the other complicating factors are relatively small. Our sensitivity analysis shows that this is also true for large SZA.
 An important point is that this energy conservation argument used to explain the good performance of algorithms like SICCS is independent of the shape of the clouds. Therefore, it also holds for horizontally inhomogeneous clouds, provided the transport of photons between pixels is disregarded, but this process is indeed relatively unimportant at the scale of SEVIRI pixels [Zinner and Mayer, 2006]. Certainly, cloud heterogeneity exerts an effect on the skill of the algorithm (Figure 11), but again this effect can only be caused by the extra error in the calculations of absorption due to the assumption of plane-parallel clouds. Figure 11 shows that the effect is relatively mild. For 90% of all samples, σBSRN is less than 0.24 (water clouds) or 0.13 (ice clouds), and hence inhomogeneity causes a change in the bias in global transmissivity less than −0.04 (water clouds) or −0.08 (ice clouds).
 Note that the energy conservation argument does not hold for direct irradiance. Consequently, under cloudy conditions, relative biases and RMSEs are much larger for direct irradiance than for global irradiance.
 One may also consider the issue of the effect of inhomogeneity from a different point of view. This is illustrated in Figure 12, which is highly schematic but demonstrates the first-order effect. Because of the nonlinear relationship with a negative second derivative between COT and TOA albedo, the COT of broken cloud fields retrieved from satellite data tends to be underestimated. This leads to relatively large biases in the optical thickness of broken cloud fields [e.g., Coakley et al., 2005]. COT and surface irradiance also have a nonlinear relationship, but here the second derivative is positive. As a result, errors due to inhomogeneity made in the inverse radiative transfer calculation of cloud properties from reflectance and in the forward calculation of irradiance from cloud properties compensate for each other to a considerable extent, so the relative difference between estimated and real transmissivity is smaller than the relative error in the estimate of COT [see also Kato et al., 2006].
 Despite the arguments put forward in the previous paragraphs, we found a remarkable systematic bias in global transmissivity in our uncorrected calculations for ice clouds (+0.0375). The almost absence of a bias for water clouds suggests errors that are specific for ice clouds. Perhaps the computed single scattering properties of the ice crystals based on assumptions about crystal habit and sizes are in error. Such errors could then lead to an underestimate of atmospheric absorption, to a systematic underestimate of the radiance in the direction of the satellite (the solar view geometry of the validation data does not have a random distribution) and/or to some narrowband-to-broadband issue. These factors would all cause a positive bias in the computed transmissivity for ice clouds. Biases may also have been introduced by mixed clouds that were incorrectly dubbed as ice clouds by CPP and by the assumed height of clouds between 1 and 2 km, which is quite unrealistic for ice clouds. In the future, these issues (ice crystal habits and sizes, mixed clouds, and cloud thickness and height) and their effect on surface irradiance should be investigated with the aim of improving the CPP/SICCS algorithm, but such studies are clearly outside the scope of the present study.
Appendix: Derivation of Continuous Surface Albedo From MCD43C3
 The MODIS MCD43C3 surface albedo product [Strahler et al., 1999], which has a temporal resolution of 8 days, is not continuous in time. This appendix describes the method used to fill the gaps.
 For each 1 × 1 km2 pixel and for overlapping periods of 16 days, the MODIS algorithm collects retrieved surface reflectivities, from which the surface albedo is determined by fitting a kernel. To obtain MCD43C3, the 1 × 1 km2 product (MCD43B3) is spatially averaged onto a climate modeling grid (CMG) with a resolution of 0.05° × 0.05°. The quality of the surface albedo on the CMG is determined in two steps. In a first step, the number of retrieved surface reflectivities that go into the kernel-fitting procedure at the 1 × 1 km2 resolution sets the quality of the retrievals at this resolution. In the second step, the qualities of all retrievals within a CMG box are combined to give the quality index of the data on the CMG. The quality of MCD43C3 generally increases with the number of overpasses during day time and the frequency of clear skies. We removed all albedo data with a quality index greater or equal to 3. Consequently, the resulting time series exhibited gaps. An example is found in Figure 13, which shows the surface albedo for a grid point in the southern Norwegian mountains. The triangles represent the original MCD43C3 samples that passed the quality test. Obviously, many data points are missing. To fill the incomplete time series (AMOD,i), we proceeded as follows:
A preliminary time series (Ai) was computed by linear interpolation of AMOD,i.
Also, the mean annual cycle was constructed from the entire time series (2000–2010) by averaging all available samples for each of the 8-day periods of the year. The computed mean annual cycle could have gaps when there was no sample at all during one or more of the 8-day periods.
These gaps were filled by linear interpolation.
A second preliminary time series (Bi), consisting of copies of the mean annual cycle for each year, was calculated. The red dashed line in Figure 13 depicts Bi for the Norwegian grid point.
The final time series (Ci) was formed by combining Ai and Bi:
where Δi is the number of time intervals of 8 days between the time of i and the time of the nearest available sample of the original time series (AMOD,i). In Figure 13, Ci is depicted by the solid black line. Using this method, all samples of the original time series were copied into the final product, whereas albedos of points in the middle of an interval of at least 160 days without any accepted retrieval were set equal to the albedo of the mean annual cycle. For other samples, a linear combination of Ai and Bi was computed.
 For the few grid points where 90% or more of the samples of AMOD,i was missing, the albedo was taken from the climatological albedo product created by Moody et al. , which was already used as the background albedo in the first version of SICCS.
 The authors thank Piet Stammes and Hartwig Deneke for their advices regarding our research and Piet Stammes for his suggestions and comments on an earlier version of this paper. They are grateful to the numerous investigators involved in the measurement and dissemination of the BSRN and AERONET data used in this paper. The investigations leading to the presented results have received funding from the European Union, Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 242093 (EURO4M).