Geophysical Research Letters

Cloud fraction errors for trade wind cumuli from EOS-Terra instruments

Authors


Abstract

[1] Errors in the standard cloud fraction products produced by the Moderate Resolution Imaging Spectroradiometer (MODIS) and the Multi-angle Imaging SpectroRadiometer (MISR) on EOS-Terra were examined using 15 m resolution data from the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER). Relative to 124 ASTER scenes containing only trade wind cumuli and having an average cloud fraction of 0.08, MODIS and MISR overestimated cloud fraction by 0.18 and 0.36, respectively. For non-sunglint scenes, MODIS and MISR overestimated cloud fractions by 0.02 and 0.24, respectively. Systematic dependences in the MODIS and MISR cloud fractions with ASTER cloud fraction were observed. Large RMS errors in MODIS and MISR cloud fractions were observed because of variations in the spatial distribution of clouds, suggesting it may be difficult to decouple long-term changes in cloud fraction from satellites from true changes in the spatial distribution of clouds.

1. Introduction

[2] Long-term satellite measurements of cloud fraction are available through programs such as the International Satellite Cloud Climatology Project (ISCCP) [Rossow and Schiffer, 1999], with potential improvements in recent years through the Earth Observing System (EOS) program (http://eospso.gsfc.nasa.gov/). The cloud fraction in the datasets available through these programs were all calculated in the same way: cloud fraction is the ratio of the number of pixels classified as cloudy to the total number of pixels. Di Girolamo and Davies [1997] refers to this as the “standard method” to calculate cloud fraction, to distinguish it from other existing methods that are generally not used to produce global cloud fraction climatologies. The pixels are classified using a cloud detection algorithm that relies on a series of threshold operations (e.g., if the visible reflectance of a pixel is greater than a predefined threshold, then it is cloudy; otherwise, it is clear), which are tailored to individual instruments and algorithms. Cloud detection results are stored into a cloud mask product.

[3] Wielicki and Parker [1992] (hereinafter referred to as WP92) provided the most thorough examination to date on the errors incurred in cloud fraction due to a combination of instrument spatial resolution and cloud detection algorithms. Based on applying a series of popular cloud detection algorithms to Landsat data at 28.5 m resolution, they concluded that systematic errors in the standard-method cloud fraction for broken, boundary layer cloud fields did not vary greatly with spatial resolution because of the competing tendency between two effects: overestimation caused by partially-filled cloud pixels that were classified as cloud and underestimation by optically thinner, partially-filled cloud pixels classified as clear. They predicted that the cloud fraction for cumulus cloud fields from ISCCP (4 km spatial resolution) would be biased high by ∼0.05, and that applying an ISCCP-like algorithm to the Moderate Resolution Imaging Spectroradiometer (MODIS; 1 km resolution) would produce cloud fraction biases less than ∼0.02. However, they did note that the statistical significance of their results is low due to only three cumulus cloud Landsat scenes analyzed. Additional Landsat scenes were further added by B. A. Wielicki et al. (Clouds and the Earth's Radiant Energy System (CERES) algorithm theoretical basis document: Overview of cloud retrieval and radiative flux inversion (subsystem 4.0), 1997, available at http://asd-www.larc.nasa.gov/ATBD) showing similar results to WP92. MODIS has since been launched on EOS-Terra and EOS-Aqua. Since the MODIS cloud detection algorithm used to produce the standard cloud fraction product is not ISCCP-like (further details in Section 2), we examine here the cloud fraction error produced by MODIS for data collected over trade wind cumuli. This is feasible through the unique space-time coincidence of the data collected by the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER; 15 m resolution) that is also on EOS-Terra. Because ASTER's spatial resolution is much smaller than the typical size of individual trade wind cumuli, cloud fraction derived from ASTER can be taken as truth, much in the same way as WP92 used Landsat at 28.5 m resolution as truth. EOS-Terra also carries the Multi-angle Imaging SpectroRadiometer (MISR), which provides a standard cloud fraction product using a different algorithm than MODIS. Cloud fraction derived from MISR is also included in our analysis.

2. Data

[4] Details of the MODIS, MISR and ASTER instruments can be found in work by Barnes et al. [1998], Diner et al. [1998] and Yamaguchi et al. [1998], respectively. Specifications of each instrument are briefly summarized in Table 1. The common region observed by the three instruments lies within the ASTER 60-km swath. ASTER archives data collected over 60 km × 60 km regions and only when tasked to do so. We used a unique set of ASTER data that was tasked over the tropical western Atlantic Ocean (20°–12°N latitude, 66°–55°W longitude) between Sept. and Dec. 2004 as part of the Rain In Cumulus over the Ocean (RICO) experiment. A full description of this dataset can be found in work by G. Zhao and L. Di Girolamo (Statistics on the macrophysical properties of trade wind cumuli over the tropical western Atlantic, submitted to Journal of Geophysical Research, 2006, hereinafter referred to as Zhao and Di Girolamo, submitted manuscript, 2006). They visually inspected each of 403 archived scenes and discarded scenes containing any amount of visible cirrus and scenes dominated by stratiform clouds in order to study the properties of trade wind cumuli. This left 152 scenes. They developed a cloud mask for each of the remaining scenes using a single, scene-dependent, manually chosen threshold applied to the 0.67 μm channel at 15 m resolution. Tabulated values of the thresholds and additional details on how these thresholds were derived can be found in work by Zhao and Di Girolamo (submitted manuscript, 2006). These cloud masks are used here.

Table 1. A Brief Summary of ASTER, MODIS and MISR Instrument Specifications
ParameterMISRMODISASTER
Number of cameras912
Surface view angle0°, ±26.2°, ±45.6°, ±60°, ±70.5°0°, −27.6°
Number of bands43614
Wavelengths, nm443–865412–14235560–11300
Swath width, km360203060
Spatial resolution, m275, 1100250, 500, 100015, 30, 90

[5] Cloud fractions from MODIS are stored on a per scene basis (Level-2) of 5 km resolution in the file named MOD06, and on a daily, eight-day, and monthly basis (Level-3) of 1° × 1° resolution in the files containing the name MOD08. Cloud fraction from MISR are stored on a per orbit basis of 17.6 km resolution in the file named CLASSIFIER, and on a 1° × 1° monthly resolution in MISR Level-3 products, which will soon be released to the public. Rather than using the coarse resolution cloud fraction products, we derive them here from the MODIS and MISR cloud masks using the same methods to produce the standard cloud fraction product. This is done to better facilitate reprojection of the data between the ASTER, MISR, and MODIS grids and to better understand the reasons for cloud fraction errors. The cloud mask for producing the MODIS cloud fractions is stored in the product file named MOD35. Versions 5 of the MOD35 was used in this study. The cloud detection algorithm used to produce MOD35 is described by Ackerman et al. [1998]. In brief, 12 multi-spectral thresholding tests are combined to classify each 1-km pixel as confidently clear, probably clear, uncertain clear, or cloudy. Both uncertain clear or cloudy are treated as cloudy in calculating cloud fraction as described by Platnick et al. [2003] and also recommended by documents available on the MOD35 webpage (http://modis-atmos.gsfc.nasa.gov/MOD35_L2/). The cloud mask for producing the MISR cloud fraction is stored in the file named RCCM. Version F04_0006 of the nadir RCCM was used in this study. The ocean component of the RCCM cloud detection algorithm is described by Zhao and Di Girolamo [2004]. In brief, thresholding tests are applied to 1.1 km 0.86 μm channel reflectance and to the standard deviation of the 4 × 4 275-m 0.67 μm channel reflectance within a 1.1 km pixel. The tests are combined to classify a 1.1-km pixel as high-confidence cloudy, low-confidence cloudy, low-confidence clear, or high-confidence clear. Both low-confidence and high-confidence cloudy pixels are treated as cloudy in calculating cloud fraction [Diner et al., 1999].

[6] There is the possibility that MOD35 and RCCM could have detected sub-visual cirrus that went undetected in the visual inspection of the ASTER scenes by Zhao and Di Girolamo (submitted manuscript, 2006). Since the focus of our study is on the cloud fraction errors for trade wind cumuli, detecting sub-visual cirrus goes against our focus. Therefore, the 152 ASTER scenes were further examined using the 1.38 μm test found within MOD35, which is very sensitive to thin cirrus [Ackerman et al., 1998]. Scenes containing cirrus fraction > 0.01 were discarded, leaving 124 scenes containing only trade wind cumuli for the analysis below.

3. Case Study

[7] We first examine a single scene to explain part of the procedure used in the statistical analysis of Section 4 and to provide a visual perception of how the cloud masks from the three instruments look like. Figures 1a and 1b show an ASTER channel 3 (0.67 μm) radiance image and its cloud mask for a scene collected on Dec. 2, 2004, centered on 17.78°N, 55.51°W. The corresponding MOD35 and RCCM are shown in Figures 1c and 1d, respectively. The cloud fraction calculated from the 15 m ASTER cloud mask (hereafter, f15) is 0.04, while the cloud fractions from the MOD35 and the RCCM (hereafter, fmod and frccm) are 0.01 and 0.11, respectively. Thus, in this case study, MOD35 is underestimating the cloud fraction by 0.03 and the RCCM is overestimating it by 0.07.

Figure 1.

(a) An ASTER channel 3 radiance image, taken on Dec. 2, 2004, centered on 17.78°N, 55.51°W, (b) the ASTER cloud mask, (c) the RCCM projected onto the ASTER domain, and (d) the MOD35 projected onto the ASTER domain. For the cloud masks, white represents cloud, grey represents clear and black represents no data.

[8] As a reference that will prove useful to compare results against in Section 4, we define a perfect clear-conservative cloud mask as one that flags a pixel as cloudy if it contains any amount of cloud, and flags a pixel as clear if it contains no amount of cloud. Such a cloud mask is known to produce large overestimates in cloud fraction, but acts as an upper bound in the cloud fraction derived from a cloud mask that misclassify clear pixels as cloudy [Di Girolamo and Davies, 1997]. To generate a perfect clear-conservative cloud mask at the pixel scales of MODIS and MISR, we projected the 15 m ASTER cloud mask onto both MODIS and MISR grids. If a MODIS 1-km pixel contains at least one 15-m ASTER cloudy pixel, then it is flagged cloudy; otherwise it is flagged clear. The same procedure is used for 1.1-km MISR pixels. The resulting perfect clear-conservative cloud mask on the MISR grid is shown in Figure 2 for the scene used in Figure 1. The perfect clear-conservative cloud mask generated on the MODIS grid is not shown, since it looks very similar to Figure 2. The cloud fractions calculated from the perfect clear-conservative cloud masks on the MODIS and MISR grids (hereafter, f1000 and f1100, respectively) are 0.41 and 0.43, which is much larger than the 0.04 of f15. However, fmod and frccm are far less than f1000 and f1100, indicating that 40% of the pixels for MOD35 and 32% of the pixels for RCCM contained optically thin, sub-pixel sized cloud that were classified as clear for this scene. This results in cloud fraction biases for MOD35 and RCCM that are much smaller than that which would be calculated using a ∼1 km resolution perfect clear-conservative cloud mask. When comparing how many pixels (clear and cloudy) from MOD35 and RCCM agree with the perfect clear-conservative cloud mask, the agreement rates are 60% and 67% for MOD35 and RCCM (hereafter, Rmod and Rrccm, respectively), for this scene.

Figure 2.

Image of the perfect clear-conservative cloud mask on the MISR grid for the same scene as Figure 1.

4. Statistics

[9] We repeated the same experiments as done in the case study to each of the 124 ASTER scenes. The average value of f15, fmod, frccm, f1000, f1100, Rmod and Rrccm for all the 124 scene are summarized in Table 2. The average cloud fraction from ASTER is 0.08. MOD35 and RCCM, on average, overestimate cloud fraction by 0.18 and 0.36, respectively. The perfect clear-conservative clouds masks, having pixel resolutions ∼1 km, overestimate cloud fraction by ∼0.4, in fairly close agreement with the RCCM.

Table 2. Average Values of f15, frccm, f1100, Rrccm, fmod, f1000 and Rmod Calculated Using the 92 Sunglint Scenes, the 32 Non-Sunglint Scenes and All 124 Scenes
 Number of Scenesf15frccmf1100Rrccm, %fmodf1000Rmod, %
Sunglint920.070.470.50830.300.4860
Non-sunglint320.100.340.49840.120.4666
Total1240.080.440.50830.260.4862

[10] Many scenes contain sunglint, where it is known that both MOD35 and RCCM may inherently have difficulties in detecting clouds (e.g., Platnick et al. [2003] (for MOD35) and Zhao and Di Girolamo [2004] (for RCCM)). These difficulties exist because the brightness of sunglint is high, which reduces the contrast between clear and cloud, and variable, which increases the probability of error when using the fixed thresholds used by the MOD35 and RCCM algorithms. WP92 did not consider cloud fraction errors that may arise from cloud detection blunders over difficult situations, such as sunglint. Thus, we stratify our results into sunglint and non-sunglint scenes. An ASTER scene is classified as a sunglint scene if its sunglint coverage > 1%; otherwise, it is a non-sunglint scene. Sunglint was conservatively set to exist within a 40° scattering-angle about the specular direction using the MISR sunglint flag in the RCCM product. These conservative criteria ensure scenes that are non-sunglint are free of sunglint effects on cloud detection. However, some scenes classified as sunglint may also be free of sunglint effects on cloud detection. For any given ASTER sunglint scene, the clear regions had fairly uniform brightness that allowed for the quality of the15-m cloud mask to be as good as the non-sunglint scenes. Statistics were produced for all the non-sunglint and sunglint scenes separately and summarized in Table 2. For sunglint scenes, MOD35 and RCCM, on average, overestimate cloud fraction by 0.23 and 0.40, respectively, while 0.02 and 0.24 for non-sunglint scenes. The perfect clear-conservative clouds masks overestimate cloud fraction by ∼0.40 for both sunglint and non-sunglint scenes. Rmod for sunglint scenes is 6% larger than that for non-sunglint scenes, while the difference in Rrccm between sunglint and non-sunglint scenes is only 1%.

[11] Figures 3a and 3b show scatter plots of fmod, frccm, f1000, and f1100 versus f15, along with ninth order polynomial fits to the data for non-sunglint and sunglint scenes. The polynomial fits to the perfect clear-conservative cloud masks in Figures 3a and 3b match remarkably well with the predictions made by Di Girolamo and Davies [1997], which were based on simulated stochastic cloud fields. Note that for some, but not all, sunglint scenes, fmod > f1000 and/or frccm > f1100, indicating that the MOD35 and/or RCCM must misclassify some clear pixels as cloudy for these scenes. However, for all the non-sunglint scenes, fmod < f1000 and frccm < f1100. Figure 3 also shows a large amount of scatter in the data. The largest contributing factor to this scatter – and the only factor for the perfect clear-conservative cloud masks – is the variability in the spatial distribution of the clouds. For example, Figure 4 gives two ASTER scenes that have widely different spatial distributions, but cloud fractions that are very close to each other: f15 = 0.09 for Figure 4a and f15 = 0.08 for Figure 4b. However, for Figure 4a, f1100 = 0.83 f1000 = 0.81, fmod = 0.12, and frccm = 0.72, while they are f1100 = 0.32, f1000 = 0.30, fmod = 0.08, and frccm = 0.21 for Figure 4b. Therefore, despite these two scenes having nearly identical cloud fractions when measured at 15 m resolution, their cloud fractions differ by ∼0.50 for the RCCM and the perfect clouds masks at ∼1 km resolution, but only 0.04 for MOD35.

Figure 3.

Scatter plots of cloud fractions from the RCCM (frccm), MOD35 (fmod), and perfect cloud masks on the MISR grid (f1100) and the MODIS grid (f1000) versus the cloud fraction from 15m ASTER cloud masks (f15), for (a) the 32 non-sunglint scenes and (b) the 92 sunglint scenes.

Figure 4.

(a) A cloud mask for an ASTER scene, taken on Sep. 22, 2004, centered on 14.45°N, 53.80°W and (b) a cloud mask for an ASTER scene, taken on Dec. 9, 2004, centered on 19.25°N, 55.71°W. White represents cloud, grey represents clear and black represents no retrieval.

5. Discussion and Conclusion

[12] We have examined the quality of the cloud fraction products produced by MODIS and MISR, which are derived from the MODIS and MISR cloud masks, namely the MOD35 product from MODIS and the nadir camera RCCM from MISR, over the tropical western Atlantic during times dominated by trade wind cumuli. Given the small sizes of these clouds, we used 124 coincident ASTER scenes to provide high-resolution (15 m) reference data for our examination.

[13] Based on the 124 scenes, MODIS and MISR overestimated cloud fraction by 0.18 and 0.36, respectively, which is remarkably large when considering the climate sensitivity to low cloud cover [e.g., Randall et al., 1984]. These biases need to be considered when using the daily and monthly mean cloud fraction products produced by MODIS and MISR for the purpose of evaluating models simulating trade wind cumuli. Moreover, we observed that the error in MODIS and MISR derived cloud fraction varied widely from scene to scene. This large variability was due to the large variability observed in the spatial distribution of trade wind cumuli. This suggests that using long-term records of cloud fraction derived from MODIS or the MISR (and likely other satellite datasets of cloud fraction) to monitor changes in cloudiness, one will need to decouple changes in the satellite-observed cloud fraction with changes in the true underlying spatial distribution of the cloud field.

[14] For the non-sunglint scenes, the average biases in cloud fraction reported here for cumulus clouds from MODIS are similar to those reported by WP92 using an ISCCP-like cloud detection algorithm, who concluded that biases in cloud fraction will be only a few percent because of the competing tendency between two effects: overestimation caused by partially-filled cloud pixels that were classified as cloud and underestimation by optically thinner, partially-filled cloud pixels classified as clear. However, we show for MODIS cloud fractions that the bias may systematically increase with cloud fraction, reaching ∼0.10 when the true cloud fraction is between about 0.20 and 0.40, as shown in Figure 3a. For the sunglint scenes, the average bias of fmod is 0.23, indicating that cloud fraction errors are not just caused by the competing tendency between the two mentioned effects, but may well be dominated by gross cloud detection errors over traditional difficult scenes, such as sunglint. The effects of sunglint are much more pronounced in MOD35 compared to RCCM.

[15] The cloud fraction products produced by MODIS are more accurate than those produced by MISR, especially outside of sunglint. This is largely because the RCCM is much more sensitive at detecting small, sub-pixel cumuli than MOD35. Indeed, the average agreement rate with the perfect clear-conservative cloud masks over the 124 scenes was 62% for MOD35 and 83% for the RCCM, indicating that the RCCM is much more clear-conservative than MOD35. Thus, if a user of the cloud fraction products wishes to know what fraction of 1 km regions contain some clouds, then the cloud fraction products from MISR are more accurate than MODIS.

Acknowledgments

[16] Partial support from the National Science Foundation under contract NSF ATM-0346172 and the Jet Propulsion Laboratory of the California Institute of Technology under contract 121756 is gratefully acknowledged. Our sincere thanks are extended to Bruce Wielicki for suggested improvements to this manuscript.

Ancillary