Toward understanding of differences in current cloud retrievals of ARM ground-based measurements



[1] Accurate observations of cloud microphysical properties are needed for evaluating and improving the representation of cloud processes in climate models and better estimate of the Earth radiative budget. However, large differences are found in current cloud products retrieved from ground-based remote sensing measurements using various retrieval algorithms. Understanding the differences is an important step to address uncertainties in the cloud retrievals. In this study, an in-depth analysis of nine existing ground-based cloud retrievals using ARM remote sensing measurements is carried out. We place emphasis on boundary layer overcast clouds and high level ice clouds, which are the focus of many current retrieval development efforts due to their radiative importance and relatively simple structure. Large systematic discrepancies in cloud microphysical properties are found in these two types of clouds among the nine cloud retrieval products, particularly for the cloud liquid and ice particle effective radius. Note that the differences among some retrieval products are even larger than the prescribed uncertainties reported by the retrieval algorithm developers. It is shown that most of these large differences have their roots in the retrieval theoretical bases, assumptions, as well as input and constraint parameters. This study suggests the need to further validate current retrieval theories and assumptions and even the development of new retrieval algorithms with more observations under different cloud regimes.

1. Introduction

[2] Treatment of clouds remains one of the largest uncertainties in current climate models [Intergovernmental Panel on Climate Change, 2007]. Improving cloud representation in climate models requires improved knowledge of cloud processes through detailed cloud observations. Cloud microphysical properties can be directly measured by in situ probes or sensors aboard research aircraft. However, aircraft data is usually only available over very limited locations and time periods due to their high associated cost. To obtain long-term continuous measurements, ground-based and space-borne remote sensors (radars, lidars, radiometers, etc.) are often used, from which cloud microphysical properties are retrieved using various algorithms.

[3] Using ground-based remote sensors and other instruments, the Department of Energy (DOE)'s Atmospheric Radiation Measurement (ARM) program has continuously monitored clouds, radiation, and the associated atmospheric states for over a decade at its primary research sites spanning latitudes from tropical to Arctic. The goal of ARM is to better understand clouds and their interaction with radiation and improve cloud parameterizations in global climate models [Ackerman and Stokes, 2003]. Various techniques [e.g., Mace et al., 1998, 2002, 2006; Wang et al., 2004; Turner, 2005; Shupe et al., 2005; Deng and Mace, 2006; Hogan et al., 2006a; Delanoë et al., 2007; Protat et al., 2007; Delanoë and Hogan, 2008; Huang et al., 2009; Dunn et al., 2011] have been developed to retrieve cloud properties from the ground-based sensors in support of ARM cloud modeling studies. While these retrieval products provide valuable information on cloud properties, large differences have been found between these cloud retrieval products in earlier studies. For example, the inter-comparison studies conducted by Comstock et al. [2007] and Turner et al. [2007a] showed large cloud property differences between different retrievals for high level ice clouds and optically thin liquid clouds, respectively; and a comprehensive focus study done by Shupe et al. [2008] demonstrated large differences that existed in several retrievals for mixed-phase clouds. These studies were based on limited cases over short time periods.

[4] Understanding the differences in current cloud retrieval products is important for evaluating and constraining climate models. In this study, we first document differences among the various cloud retrieval products and then try to understand what differences can be explained by the differences in the basic inputs and constraints or by the differences in the theoretical basis and assumptions used in the retrieval algorithms. It is important to understand the distinction between these retrieval elements. Here the theoretical basis denotes the retrieval algorithm instrument basis and cloud property derivation methods, such as a radar-based parameterization or a radiance-based optimal estimation. Assumptions are those simplified conditions applied to the complex cloud systems during retrievals, like the particle size distribution and ice particle habit. Inputs are those measurements required by the cloud retrieval algorithms in order to derive the cloud properties. Constraint parameters are those additional measurements that can be used to further improve the quality of retrieved cloud properties. Our goal is to examine the differences among the available cloud retrievals, point out the potential causes from the way they are retrieved, and explore potential issues that need to be addressed in the future. In contrast to earlier studies, our analysis is performed over a much longer time period and emphasizes two different cloud regimes so that statistical characteristics of the examined cloud retrievals for these two cloud regimes can be explored. Nine ground-based cloud retrieval products that are available over multiple years at the ARM Southern Great Plains (SGP), North Slope of Alaska (NSA), and Tropical Western Pacific Sites (TWP) are analyzed.

[5] In section 2, we briefly describe the nine ground-based cloud retrieval products used in this study. Section 3 shows how the different cloud properties retrieved from various algorithms are affected by their algorithm differences, as well as differences in their input and constraint parameters. In section 4, a statistical analysis based on multiyear data is carried out to illustrate the systematic differences between cloud retrieval products. A summary of findings and a brief discussion of future studies are given in section 5.

2. Ground-Based Cloud Retrievals

[6] Table 1 lists the nine ground-based cloud retrievals along with their principal investigator (PI) affiliations and primary references. For each of the five ARM permanent research sites, i.e., SGP, NSA, TWP Manus Island (TWPC1), TWP Nauru Island (TWPC2), and TWP Darwin (TWPC3), there are multiple cloud retrieval products. Note that not all of them are available for all the sites and all types of clouds. The exception is the MICROBASE product, which is currently the ARM baseline cloud retrieval value-added product (VAP) and contains all cloud properties for all cloud conditions over the five sites. The primary retrieved cloud microphysical products are liquid water content (LWC) and liquid effective radius (re) for liquid clouds and ice water content (IWC) and ice re for ice clouds for almost all the cloud retrieval products except for CLOUDNET which only has retrievals of LWC and IWC. Table 2 shows the estimated uncertainties in these cloud products indicated by the algorithm developers in their publications. For most retrieval products, the uncertainties in liquid water path (LWP), LWC, liquid re, IWC, and ice re are about 20–30%, 10–100%, 10–60%, 10–100%, and 10–50%, respectively. However, it should be noted that these provided uncertainties were usually estimated based on a limited number of observations over limited periods. In addition, it represents the uncertainty associated with a specific algorithm rather than overall uncertainty in the retrieved property. Previous studies [Comstock et al., 2007; Turner et al., 2007a] have shown cases that the difference between retrieved properties from different algorithms is often much larger than the uncertainty associated with a specific algorithm.

Table 1. Nine Ground-Based Cloud Retrieval Products at Five ARM Sites Along With Their Primary Microphysical Products, PI Contact Information and Referencesa
ProductsMicrophysical ProductsContact PIsAffiliationsSitesCloudsReferences
  • a

    CAWCR indicates ‘the Centre for Australian Weather and Climate Research’; and LATMOS indicates ‘the Laboratoire ATmosphère, Milieux, Observations Spatiales’.

MICROBASELWC, liquid re IWC, ice reMike Jensen; Maureen DunnBrookhaven National LabAll 5 sitesLiquidLiao and Sassen [1994]; Frisch et al. [1995]
IceLiu and Illingworth [2000]; Ivanova et al. [2001]
MixedAll above
MACELWC, liquid re IWC, ice reGerald MaceUniversity of UtahSGPBoundary StratusDong et al. [1998] (Layer); Dong and Mace [2003] (Profile)
Other LiquidFrisch et al. [1998]
CirrusMace et al. [1998] (Layer Average); Mace et al. [2002] (Vertical Profile)
Other IceLiu and Illingworth [2000]
CLOUDNETLWC, IWCRobin Hogan; Ewan O'ConnorUniversity of ReadingSGP, TWPC3Liquid partIllingworth et al. [2007]
Ice partHogan et al. [2006a]
DENGIWC, ice reMin DengUniversity of WyomingAll 5 sitesCirrusDeng and Mace [2006]
SHUPE_ TURNERLWC, liquid re IWC, ice reMatthew Shupe; David TurnerUniversity of Colorado, NOAA National Severe Storms LabNSALiquid onlyFrisch et al. [1995]; Turner et al. [2007b]; Turner [2007]
Liquid and ice in thin cloudsTurner [2005, 2007]; Turner et al. [2007b]
Ice partShupe et al. [2005]
WANGLWC, liquid re IWC, ice reZhien WangUniversity of WyomingNSAMixedWang and Sassen [2002] Wang et al. [2004]; Wang et al. [2007]
COMBRETLWC, liquid re IWC, ice reJenifer ComstockPacific Northwest National Lab3 TWP sitesLiquid (radar)Same as MICROBASE
Ice (radar + lidar)Wang and Sassen [2002]
Ice (radar only)Hogan et al. [2006a]
Ice (lidar only)Comstock and Sassen [2001]; Fu [2007]
Drizzle, rainWood [2005]
RADONIWC, ice reAlain Protat; Julien DelanoëCAWCR, LATMOSTWPC3IceDelanoë et al. [2007]
VARCLOUDIWC, ice reAlain Protat; Julien DelanoëCAWCR, LATMOSTWPC3IceDelanoë and Hogan [2008]
Table 2. Approximate Uncertainties in the Major Cloud Microphysical Properties Derived From Different Retrievals in Nine Cloud Products Based on Literaturea
ProductsCloudsCloud Property Uncertainties From Literature
LWPLWCLiquid reIWCIce re
  • a

    “NG” indicates that the uncertainty is not given in the literature and “-” denotes that the variable does not exist or is not retrieved for the specific cloud type.

MACEStratus (layer)∼25%NG∼16%--
Stratus (profile)∼20–30%∼20–100%∼20–60%--
Other Liquid∼20–30%∼15%NG--
Other Ice---∼30–50%NG
CLOUDNETLiquid part∼20–30%NG---
Ice part---∼30–50%-
SHUPE_ TURNERPure liquid clouds∼20–30%∼10–50%∼10%--
Liquid and ice in optical thin clouds∼20–30%∼40–70%∼20–30%∼60–100%∼20–50%
Ice in other clouds---∼60–100%∼20–50%
COMBRETLiquid (radar)Same as MICROBASE
Ice (Ze and σext)---∼10–100%∼10–30%
Ice (Ze)---∼30–50%NG
Ice (σext)---NGNG
Drizzle and Rain---NGNG
VARCLOUDIceUncertainties are dependent on errors in measurements and accuracy in forward models

[7] These cloud retrieval products have been widely used in various studies to validate and improve cloud microphysical parameterizations in climate and weather forecast models as well as to evaluate satellite observations. For example, Xie et al. [2005] and Xu et al. [2005] used the MICROBASE data to examine model simulated cloud microphysical properties of midlatitude frontal clouds in multimodel intercomparison studies. Klein et al. [2009] used both SHUPE_TURNER and WANG cloud products for assessment of model simulations of single-layer mixed-phase clouds in the Arctic. Illingworth et al. [2007] and Bouniol et al. [2010] applied the CLOUDNET process to assess numerical weather forecast models. Dong et al. [2008] used the MACE retrieval products to assess satellite cloud remote sensing.

[8] The nine cloud retrieval products are mainly based on the measurements from the millimeter-wavelength cloud radar (MMCR), micropulse lidar (MPL), microwave radiometer (MWR), radiosonde soundings, and the atmospheric emitted radiance interferometer (AERI). These instruments provide measurements of water equivalent radar reflectivity (Ze), Doppler velocity (Vd), spectral width (σd), lidar backscatter coefficient (β), cloud top and bottom locations, cloud particle visible extinction coefficients (σext), LWP, temperature, and spectral radiance, which are used as either critical inputs or constraints in various retrieval techniques. We want to emphasize here that these cloud retrieval products have used different MWR measurements of LWP. For example, MACE, SHUPE_TURNER, and COMBRET have used the instantaneous LWP retrieved by Turner et al. [2007b], MICROBASE has used the statistical LWP retrieved by Turner et al. [2007b], CLOUDNET has used the LWP retrieved with the method developed by Gaussiat et al. [2007], and WANG has used the LWP retrieved by Wang [2007].

[9] For different cloud regimes measured by various remote sensors, the cloud retrieval algorithms vary widely. In particular, different algorithms are typically used to retrieve cloud properties based on the cloud phase. Even for the same remote sensing measurements, the cloud retrievals could be different because of their different assumptions. Table 3 provides a summary of the retrieval methods used in the nine ground-based cloud retrieval products, including the cloud types to which the retrieval algorithms are applied, the algorithm and its retrieval theory basis and major assumptions, as well as the basic inputs and constraints used in the retrieval techniques. Brief explanations of the symbols and abbreviations in Table 3 are given in the notation section. Main features of the nine retrieval products are presented in a technical report [C. Zhao et al., 2011].

Table 3. Retrieval Algorithms (Assumptions, Retrieval Ideas, Functions and Inputs) for Nine Cloud Productsa
ProductsCloudsMethodTheory Based Functions/Models/ParametersAssumptionsMajor InputsConstraints
  • a

    The meanings of the symbols and abbreviations are listed in the notation section.

MICROBASELiquidEPMLWC = F(Ze, LWP); re = F(Ze, LWC); N ∼ 200 cm−3Lognormal (σ = 0.35)sphericalZeLWP
IceEPMIWC = F(Ze); re = F(T)ExponentialPlanar polycrystalZe, T 
MixedEPMfice = −T/16; Zeliquid = (1 − fice)*Z; Zeice = fice*ZSee aboveSee aboveZe, TLWP
MACEBoundary stratus (layer)EPM; optimalThick: re_layer = F(LWP, γ, μ0); Thin: δ-2 stream modellognormal (σ = 0.35)SphericalLWP, γ, μ0LWP
Boundary stratus (profile)ForwardLWC = F(LWP, Ze); day: re = F(re_layer, Ze); night: re = F(Ze)LognormalSphericalLWP, ZeLWP
Other LiquidForwardLWC = F(LWP, Ze);<r6 > = < r3 > 2-sphericalLWP, ZeLWP
Cirrus (layer)OptimalMODTRAN3 model (optical thin)Modified Gamma (α = 1)hexagonalZe, I 
Cirrus (Profile)ForwardZe = F(L, n(L)); Vd = F(L, n(L), V(L)); σd2 = F(L, n(L), V(L))ExponentialBullet RosetteZe, Vd 
Other IceEPMIWC = aZeb, a, b are constantsExponential-Ze 
CLOUDNETLiquid partForwardLWC from LWP-scale with adiabatic gradient--T, P; LWPLWP
Ice partEPMIWC = F(Ze, T)-spherical aggregatesT, P; Ze 
DENGIceOptimalZe = F(λ, N0); Vd = F(λ, Wm); σd = F(λ, Wσ); Wσ = F(σd, Ze);ExponentialhexagonalZe, Vd, σd 
SHUPE_ TURNERPure liquid cloudsForwardre = F(Ze, N) with adjusted N; LWC = F(Ze)LognormalSphericalZe, LWPLWP
Liquid and ice in optical thin cloudsOptimalLiquid and ice re: AERI based LWC: adiabatic gradient scaled by LWP; IWC = aZebGammaAnyI; LWP, ZeLWP
Ice in other cloudsEPMIWC = aZeb; re = F(Ze); a = a(time), b = 0.63exponential-Ze 
WANGMixedForward OptimalIce part: IWC = F(σext, re); re = F(σext, Ze); Liquid part: DISORT;Modified gamma; lognormalhexagonalLWP, I, Ze, σext, TcbLWP
COMBRETLiquid (radar) Same as MICROBASE, except N = 100 cm−3
Ice (Ze and σext)EPMIWC = F(σext, Ze); re = F(σext, Ze);Modified GammahexagonalZe, σext 
Ice (Ze or σext)EPMIWC = F(Ze, T); IWC = F(σext, T); re = F(IWC, Ze); re = F(IWC, σext)Fitting Gamma-Ze, T or σext, T 
Drizzle and RainEPMR = F(Ze); N(r) = F(R, r); re = volume/area;Marshall-Palmer type-Ze 
RADONIceForwardIWC = f(Ze, N0*), σext = f(Z, N0*), Dm = f(VT), re = F(IWC, σext);Normalized (N0*, Dm)retrievedZe, Vd 
VARCLOUDIceOptimalRadar and lidar forward models. (IR forward model available)Normalized (N0*, Dm)spherical aggregatesZe, σext, I, T 

[10] The nine retrieval products differ from each other in many ways, such as the cloud phase classification, cloud masks, Ze calculation and threshold values used in the algorithm, and the treatment of drizzle. For example, MICROBASE uses a simple temperature-based phase classification method in which the clouds are classified as liquid, mixed or ice phase for the temperature (T) range of T ≥ 0°C, −16°C < T < 0°C, and T ≤ −16°C, respectively, while SHUPE_TURNER and COMBRET use an advanced cloud phase classification method developed by Shupe [2007] which is based on the combination of radar, lidar, LWP and temperature. For Ze, most retrieval products use the value-added product of the Active Remotely Sensed Cloud Locations (ARSCL) [Clothiaux et al., 2000] while MACE and CLOUDNET do their own radar moments processing methods. This will result in slightly different Ze values used in these retrievals. For the treatment of drizzle, some retrieval products (e.g., COMBRET) have classified drizzle from clouds while others just flag the presence of drizzle (e.g., MICROBASE). In reality, drizzle is present in many low-level warm (particularly thick) clouds [Kubar et al., 2009]. Furthermore the combination of radar and lidar can see more ice clouds than radar only [Borg et al., 2011]. All these differences are examples of differences in the cloud retrieval inputs.

[11] In practice, various assumptions are made within these retrieval algorithms, including assumptions about the particle size distributions (PSD), ice crystal habit, and ice density (ρi). Different assumptions are often used because of high natural variability of cloud properties and different interpretations of in situ observations. As shown in Table 3, all retrieval algorithms assume a lognormal PSD for liquid particles but they assume either an exponential or a (modified) gamma PSD for ice particles. The assumption for ice crystal habit varies between different retrievals and different locations. Most radar-based retrieval algorithms also assume applicability of Rayleigh scattering theory, or the (6 + κ)th power law relationship between particle size and Ze when the radar wavelength is large compared to the scatterers. Here κ is a parameter dependent on ice crystal habit and ice bulk density.

[12] For ice re, the definition might be different for the cloud products with non-spherical ice crystal habit assumption [McFarquhar and Heymsfield, 1998], and we need to convert them into the same definition for the following intercomparison studies. For the nine cloud products studied here, MICROBASE, SHUPE_TURNER, DENG, RADON, and VARCLOUD have used the definition of

display math

where ρi and Ac are solid ice density (0.92 g cm−3) and projected area associated with the size distribution. Note that DENG, RADON and VARCLOUD directly use this equation with their derived ice extinction coefficient while MICROBASE and SHUPE_TURNER uses a T-based and a Ze-based parameterization method, respectively. MACE has used the effective spherical radius defined in terms of the total volume of the distribution to the total area [Mace et al., 1998] and we assume it is comparable to the definition of equation (1) in this study. Differently, COMBRET and WANG use a generalized effective diameter Dge [Fu, 1996]. In this study, we convert Dge to the ice re defined in equation (1) using an equation [Fu, 1996, equation (3.12)]

display math

As described in Fu [1996], the error caused by this conversion is small.

[13] The current data availability for the nine cloud retrieval products using ARM measurements is shown in Figure 1. There are three to six different retrieval products available for the study period which runs from 2002 to 2007 at SGP, NSA, TWPC1 and TWPC2, and from 2005 to 2008 at TWPC3. To facilitate the intercomparison, all the retrievals have been converted to a uniform format with hourly time resolution and 45 m vertical resolution. The same cloud samples (same height and time) are used for comparing different retrieval products.

Figure 1.

Data availability for the nine ground-based cloud retrieval products between 1997 and 2009 at five permanent ARM research sites. MICROBASE, MACE, SHUPE_TURNER and COMBRET are for all cloud properties; CLOUDNET is for LWC and IWC of all clouds; DENG, VARCLOUD and RADON are for ice cloud properties only; and WANG is for mixed-phase cloud properties only.

3. Differences Between Cloud Retrievals

[14] Stratus and cirrus clouds are the two types of clouds that most ground-based retrieval techniques are developed specifically for due to their radiative importance and relatively simple structures. Even for these two types of clouds, however, earlier case studies showed that large differences exist in retrieved cloud properties among various retrieval algorithms [Turner et al., 2007a; Comstock et al., 2007]. Different from these past case studies, this study targets understanding the differences between cloud retrieval products from their algorithm details in a statistical way. For simplicity, we examine only single-layer boundary layer overcast clouds and high level ice clouds. The boundary layer overcast clouds are defined as those single layer liquid and mixed-phase clouds with cloud top below 2 km and hourly cloud fraction over 90%. Note that 2 km is selected to make sure that most of the selected “boundary layer clouds” are stratiform in nature. We realize that this fixed, cloud top threshold may miss some of the deepest clouds given the seasonal and geographical variability in boundary layer depth. A sensitivity test using a 3 km threshold at the TWP sites yields similar results to those found when using 2 km. The high level ice clouds refer to single-layer ice clouds with hourly cloud fraction over 90% and cloud bases above 4 km, 5 km and 6 km at NSA, SGP and TWP, respectively. The cloud boundary, cloud layer and cloud fraction used to identify these two types of clouds are from the ARM climate modeling best estimate (CMBE) [Xie et al., 2010], which is based on cloud frequency of occurrence from the vertically pointing MMCR and MPL. In what follows we will show the cloud retrieval differences based on the retrieval results during the period between May and November in 2004 at NSA and SGP and in 2007 at TWPC3.

3.1. Boundary Layer Overcast Clouds

[15] As shown in Table 3, the retrieval techniques usually differ from each other in their basic inputs, fundamental theories, and assumptions. Below we try to understand how these differences impact the retrieved cloud liquid properties in both liquid-phase and mixed-phase clouds.

3.1.1. Liquid Phase Clouds

[16] Boundary layer clouds are generally liquid phase at the SGP and TWP sites while mixed-phase at the NSA site. For pure liquid phase clouds, there are five retrieval products, MICROBASE, MACE, CLOUDNET, SHUPE_TURNER, and COMBRET. These cloud products use either an optimal estimation method or empirical parameterization method.

[17] Figure 2 shows the probability distribution of cloud bases and LWP for the same boundary layer liquid clouds in MICROBASE, MACE and CLOUDNET at SGP for period of May through November 2004. Clear differences exist in both cloud bases and LWP among these three cloud products. Given the fact that these quantities are used either as the basic inputs (e.g., cloud bases) or constraints (e.g., LWP), the differences shown in Figure 2 inevitably lead to discrepancies in the retrieved cloud properties such as LWC. This is illustrated in Figures 3a and 3b, which show the vertical distributions of the mean LWC for the same clouds retrieved by MICROBASE, MACE, and CLOUDNET at SGP, and by MICROBASE and SHUPE_TURNER at NSA, respectively. The large offset in amount of retrieved LWC between MICROBASE and MACE or between MICROBASE and CLOUDNET at SGP (Figure 3a) indicates that the constrained MWR LWP used in MICROBASE is larger than those used in MACE and CLOUDNET. This is consistent with the averaged LWP values for this period, which are 149.15, 117.04 and 108.54 g/m2 for MICROBASE, MACE and CLOUDNET at SGP. The similar amount of retrieved LWC between MICROBASE and SHUPE_TURNER shown in Figure 3b is also consistent with their constrained LWP, which are 70.12 and 68.27 g/m2, respectively.

Figure 2.

Probability distribution functions of boundary layer liquid cloud bases and LWP from three cloud retrieval products at SGP for period between May and November in 2004.

Figure 3.

The differences of vertical distributions of mean LWC at each height for the same boundary single layer overcast clouds (liquid phase) from May through November in 2004 (a) between MICROBASE, MACE and CLOUDNET at SGP, and (b) between MICROBASE and SHUPE_TURNER at NSA.

[18] It should be noted that even constrained by the same LWP, the vertical structure of the retrieved LWC can still be different due to the differences resulted from different algorithm theory bases. For example, the vertical gradients of LWC are proportional to Ze1/1.8 in MICROBASE, COMBRET and SHUPE_TURNER (liquid only clouds), and proportional to Ze1/2 in MACE, while they follow an adiabatic gradient determined based on temperature and moisture profiles in CLOUDNET and SHUPE_TURNER (mixed-phase clouds). The LWC vertical distributions shown in Figure 3 demonstrate these theory-related differences (MICROBASE, MACE and CLOUDNET) and similarities (MICROBASE and SHUPE_TURNER).

[19] For liquid re, both MACE and SHUPE_TURNER use a radiance based optimal estimation method for optical thin clouds and a Ze-based parameterization method for optical thick clouds. The radiance based optimal estimation method makes use of the optimal match of surface shortwave or infrared (IR) radiance between measurements and calculations. The radar based parameterization method makes use of the 6th power law relationship between droplet size and Ze, which most heavily weights the large droplets. In contrast, liquid re in MICROBASE and COMBRET is obtained using the power law relationship between LWC and liquid re, an assumed constant cloud number concentration (N), and a lognormal particle size distribution. There is a notable difference for the derivation of liquid re between MICROBASE and COMBRET. MICROBASE derives cloud liquid re using scaled LWC by MWRRET LWP and N = 200 cm−3 for all sites, while COMBRET calculates liquid re using LWC before being scaled by the MWRRET LWP and N = 100 cm−3 for TWP sites. N = 200 cm−3 in MICROBASE is generally a reasonable assumption for land area at SGP [Miles et al., 2000]. However, for NSA and TWP sites, this number is likely too large and will result in an underestimation of liquid re. In comparison, N = 100 cm−3 used in COMBRET for TWP sites is a more reasonable assumption since the clouds are more maritime in origin [McFarlane et al., 2002].

[20] Figures 4a, 4b and 4c show the differences in hourly averaged liquid re for the same liquid-only clouds between MACE and MICROBASE at SGP, between SHUPE_TURNER and MICROBASE at NSA, and between MICROBASE and COMBRET at TWPC3, respectively. Figure 4a shows slightly smaller liquid re in MICROBASE than MACE at SGP, which is most likely due to the differences in their retrieval basis. The radar reflectivity-based retrieval used in MACE more heavily weights the larger droplets compared to the LWC-based retrieval in MICROBASE, resulting in a much higher occurrence of larger 15–20 um liquid re as shown in Figure 4a. Figure 4b shows that liquid re in MICROBASE is systematically less than that in SHUPE_TURNER. Considering they have similar LWC (Figure 3b), this difference is caused by the larger number concentration assumption used in MICROBASE at NSA. Note that the peak shown in the SHUPE-TURNER line is due to the climatological value (8 μm) assumed for re when other techniques cannot be used. In contrast, Figure 4c shows a similar re distribution between MICROBASE and COMBRET since they use similar retrieval algorithms. The slight difference in cloud liquid re between MICROBASE and COMBRET must be related to a combination of their differences in the LWC used for liquid re calculation and the assumption in droplet number concentration.

Figure 4.

The retrieval difference of liquid re for exactly the same boundary single layer overcast clouds during the period of May through November for (a) MICROBASE and MACE in 2004 at SGP, (b) MICROBASE and SHUPE_TURNER in 2004 at NSA, and (c) MICROBASE and COMBRET in 2007 at TWPC3.

[21] Figure 5 shows the relationship between LWC and liquid re for liquid cloud products at SGP, NSA and TWPC3. Note the red lines in Figure 5 are the fitting lines with a 2nd order polynomial function. The LWC-re relationship varies among the retrieval products. Theoretically,

display math

where ρl is water density, r is droplet size, and N(r) is droplet size distribution. With an assumed particle size distribution, if cloud droplet number concentration (N) is assumed (MICROBASE and COMBRET), LWC and liquid re follow a power law relationship. If LWC and re are derived independently (MACE, SHUPE_TURNER) with no assumption on N, the two variables might show unexpected relationships. This feature is clearly illustrated in Figure 5, which shows LWC and liquid re follow a good power law relationship in MICROBASE while demonstrate a poor correlation with almost no relationship in MACE and SHUPE_TURNER. COMBRET (Figure 5f) shows a much weaker power relationship between LWC and liquid re compared to that for MICROBASE. Although COMBRET uses a similar retrieval algorithm as MICROBASE, it calculates liquid re using LWC before it is scaled by the MWRRET LWP. In other words, the final scaled LWC is no longer consistent with the LWC used to derive liquid re, which is why the power relationship in COMBRET is weak. In principle, the algorithms used to derive liquid re in MACE and SHUPE_TURNER are more physically based than those used in MICROBASE and COMBRET, which just simply assume a constant liquid N.

Figure 5.

The relationship between LWC and liquid re for different retrieval products of pure liquid clouds during the period of May through November in 2004 at SGP and NSA, and in 2007 at TWPC3. The red lines are the fitting lines with a 2nd order polynomial function.

3.1.2. Mixed-Phase Clouds

[22] Mixed-phase clouds are frequently observed at NSA [Shupe, 2011] and their liquid component has a large impact on cloud radiative effects [Shupe and Intrieri, 2004]. Below we discuss the differences among mixed-phase cloud microphysical properties retrieved at the ARM NSA site.

[23] Associated with the retrieval theoretical basis, clear differences exist in the vertical variations of cloud properties. As shown in Table 1, there are 3 cloud retrieval products for boundary layer mixed-phase clouds at NSA, which are MICROBASE, SHUPE_TURNER and WANG. For the following vertical variation analysis, WANG is not discussed since it represents layer averaged cloud properties. MICROBASE obtains the cloud liquid re based on Ze and cloud temperature (T) using

display math
display math
display math

where Zeliq, σx and Δz are water equivalent radar reflectivity contributed by liquid droplets, spectral width of cloud droplet lognormal size distribution, and the length of each radar range gate, respectively. T is between −16 and 0°C. In contrast, SHUPE_TURNER derives the layer averaged cloud liquid re using AERI-based optimal estimation method for optical thin clouds, derives the profiles of cloud liquid re using Ze-based parameterization method for all-liquid layers, and sets a climatology value of 8 μm for those that cannot be retrieved.

[24] Figures 6a and 6b show the vertical structure of hourly averaged liquid re from MICROBASE and SHUPE_TURNER for mixed-phase clouds. Differences in cloud phase classification can be clearly seen between MICROBASE and SHUPE_TURNER. Since the cloud temperature ranges from −19°C to 0.5°C (most around −9°C) for this period, MICROBASE classifies almost all clouds/hydrometers as mixed-phase. Differently, SHUPE_TURNER determines the clouds based on radar and lidar, not on cloud temperature. This is the reason that MICROBASE has classified many mixed-phase clouds below 400 m in October 2004 while SHUPE_TURNER has not. This is a clear case indicating the importance of accurate and consistent cloud phase classification.

Figure 6.

Difference in vertical structure of liquid re between (a) MICROBASE and (b) SHUPE_TURNER in October 2004 at NSA site.

[25] Algorithm-related differences in cloud properties can also be found in Figure 6. As shown in Figure 6a, the decrease of cloud temperature with height results in a decrease of liquid re with height in MICROBASE for period between October 9 and October 15, 2004. In contrast, the hourly average of cloud liquid re from SHUPE_TURNER generally increases or stays constant with height within a layer, particularly for the upper layer of clouds (Figure 6b). Figure 6b also shows that SHUPE_TURNER algorithm has a prominent feature with climatological value of 8 microns throughout most of the cloud, which is potentially a significant limitation for this product. Over the same period, aircraft measurements in the Arctic from the Mixed-Phase Arctic Cloud Experiment (M-PACE) have shown a typical vertical structure of single-layer mixed phase clouds in which the liquid re increases with height [Verlinde et al., 2007]. This feature has also been observed in other field campaigns in the Arctic region, like the First ISCCP (International Satellite Cloud Climatology Project) Regional Experiment/Surface Heat Budget of the Arctic (FIRE-ACE/SHEBA) [Hobbs et al., 2001]. These observed features in mixed-phase clouds clearly cannot be captured by a T-dependent cloud phase partition scheme [Zhao and Wang, 2010] such as that used in MICROBASE, indicating the serious problem associated with the MICROBASE retrieved cloud properties for mixed-phase clouds.

[26] The retrieved cloud liquid microphysics, particularly the cloud liquid re, also exhibits a notable difference in the probability density functions. As an example, Figure 7 shows that the liquid re retrieved from both SHUPE_TURNER and WANG is systematically larger than that from MICROBASE for single-layer, mixed-phase boundary layer clouds. As indicated earlier, the large droplet number concentration assumed in MICROBASE might make liquid re underestimated at NSA. Another possible reason is that the estimation of Ze for the cloud droplets in equation (4) as MICROBASE uses has little validity, causing the retrieved liquid re to be systematically smaller than the others. In other words, liquid re from SHUPE_TURNER and WANG are likely more reliable than that from MICROBASE, particularly for mixed-phase clouds.

Figure 7.

The difference in the cloud liquid re for the same single-layer, mixed-phase boundary layer clouds between retrievals from MICROBASE, SHUPE_TURNER, and WANG in May through November 2004 at NSA. Note that Liquid re at different vertical layers has been set as the layer averaged value in the cloud product of WANG.

[27] There are some other assumptions made in current retrievals of boundary layer overcast cloud properties, such as the horizontal homogeneity assumption and the lognormal particle size distribution (PSD) assumption. These assumptions also introduce uncertainties in the cloud retrievals. However, since these assumptions are similar for all the retrieval algorithms examined in this study except for their difference in the time resolution, they cannot be the main reason for the large differences found in these retrievals and are not discussed here.

3.2. High Level Ice Clouds

[28] Ice cloud retrieval techniques can also be classified into two categories: the forward or optimal estimation approach (MACE, DENG, VARCLOUD, and RADON) and the empirical parameterization method (MICROBASE, MACE, CLOUDNET, SHUPE_TURNER, and COMBRET). Note that the MACE cloud product includes retrievals from both categories. The forward approach uses theoretically based equations with certain assumptions (forward models) to derive cloud properties. In contrast, the empirical parameterization method uses empirical regression equations derived from aircraft observations. In general, there are more unknowns and therefore more assumptions that need to be made for ice clouds than liquid clouds due to their complexity in bulk density, ice crystal habit, and particle formation processes. Moreover, the retrieval of ice cloud properties is hampered by a lack of constraints on the total IWP. These extra limitations could result in larger retrieval uncertainties for high level ice clouds than for boundary layer overcast clouds.

[29] We first emphasize the cloud retrieval differences related to the fundamental basis of the retrieval algorithms. It is seen from Table 3 that the high level ice cloud properties are derived using radar-based retrieval methods by all retrieval algorithms. Some of them also use the spectral radiance (MACE, SHUPE_TURNER) or lidar extinction coefficient (COMBRET, VARCLOUD). Note that results from WANG are not presented in this section since the product currently only includes mixed-phase cloud properties.

[30] Figure 8 shows large discrepancies in retrieved ice cloud properties, which are highly related to the algorithm basis. For all ARM sites, ice re from MICROBASE is generally smaller with a narrower range than that from others. This is mainly because ice re in MICROBASE is retrieved based on cloud temperature using [Ivanova et al., 2001]

display math
Figure 8.

Differences in retrieved ice re between the ground-based retrieval products for high level ice clouds during the period between May and November in 2004 at (top) SGP and (middle) NSA, and (bottom) in 2007 at TWPC3.

[31] At the maximum 0°C for ice clouds, MICROBASE has a maximum re of 37.7 μm, demonstrating a very limited range. On the other hand, MACE, DENG, and SHUPE_TURNER derive ice re by making use of the (6 + κ)th power relationship between cloud particle size and Ze. In general, ice re in MICROBASE is less than those retrieved from radar reflectivity. Interestingly, ice re from DENG at NSA and from RADON at TWPC3 is even smaller than that from MICROBASE. The small ice re in DENG at NSA might be associated with the parameters (e.g., particle mass-length relationship) used for clouds at NSA, which has not been explicitly evaluated and might have relatively large uncertainties. The small ice re in RADON at TWPC3 is likely due to the fact that the statistical relationship between ice particle density and maximum dimension is retrieved for each cloud from the fall speed – Ze relationship rather than assumed the same for all clouds as in other methods. To know which algorithm is more valid requires further comparisons with accurate in situ observations.

[32] Cloud property differences associated with the fundamental basis used in the ice retrievals can also be illustrated by the relationship between IWC and ice re. The IWC and ice re from MACE, SHUPE_TURNER and DENG are both derived from Ze-based parameterization methods. In contrast, IWC and ice re in MICROBASE are derived using Ze-based and T-based parameterization methods, respectively; and they are related through the visible extinction coefficient using an optimal estimation method in VARCLOUD. Correspondingly, Figure 9 shows a clear power law relationship between IWC and ice re for MACE, SHUPE_TURNER, and DENG, and a relatively weaker power law relationship for MICROBASE and VARCLOUD for high level ice clouds. Interestingly, IWC in MICROBASE generally increases with ice re but roughly decreases at larger ice re, particularly at TWPC3. This is most likely due to the misclassification of cloud phases based on cloud T. If the clouds becomes mixed-phase at cold T around −30°C to −20°C, Ze and IWC will most likely decrease while ice re still increases according to the T-based algorithm, leading to the features shown in Figure 9. Similar to MICROBASE, COMBRET and RADON also demonstrate that IWC increases first and then decreases with ice re at TWPC3. The IWC-re relationship for COMBRET is likely caused by multiple scattering in the lidar signal near cloud top, resulting in an overestimation of the extinction. It has been found that when the optical depth is smaller and the multiple scattering is less, IWC is positively correlated with ice re in COMBRET. But for thicker clouds, which tend to dominate in the tropics, negative correlation between IWC and ice re often resulted from the lidar extinction near cloud top. The reasons for the IWC-re relationship in RADON need further investigation.

Figure 9.

Relationships between IWC and ice re for high level ice clouds between different retrievals for period of May through November in 2004 at SGP and NSA and in 2007 at TWPC3.

[33] We next examine how cloud retrieval differences are caused by uncertainties in defining various empirical parameters used in these algorithms. The regression equations and empirical parameters are often derived based on limited aircraft in situ measurements, which may not be valid globally due to the complexity of clouds. The different parameters used by various retrieval algorithms will cause discrepancies in the retrieved cloud properties. For example, many ice cloud retrieval algorithms use IWC = aZeb to determine IWC. However, parameters a and b are defined differently in different retrieval techniques. In MICROBASE and MACE, a = 0.097 and b = 0.59. In SHUPE_TURNER, the parameter a is a tunable parameter dependent on time of year and roughly lies between 0.05 and 0.12 and b = 0.63. Note that some of the differences in parameters a and b are due to latitudinal dependence of these coefficients. These parameter differences will impact retrieval results where, for instance, they can make IWC from SHUPE_TURNER systematically less than that from MICROBASE at NSA.

[34] Next, we discuss the retrieval differences associated with ice crystal habit assumptions. An ice crystal's habit here refers to its visible external shape. As indicated by Comstock et al. [2007], for each ground-based ice cloud retrieval technique examined, an assumption concerning the ice crystal habit has to be made. For example, the ice crystal habit is assumed to be hexagonal column [Mace et al., 1998] and bullet rosette [Mace et al., 2002] in MACE, planar polycrystals in MICROBASE, and hexagonal columns in WANG, COMBRET and DENG. However, various field observations and lab studies [McFarquhar and Heymsfield, 1996; Korolev et al., 1999; Heymsfield and Iaquinta, 2000; Noel et al., 2004; Verlinde et al., 2007; McFarlane and Marchand, 2008; Bailey and Hallett, 2009] have shown high geographic and temporal variability of ice crystal habits. Because various ice crystal habits often result in different relationships between particle mass, ice bulk density, or terminal velocity and particle maximum dimension length, the necessary but simplified ice crystal habit assumptions can have a large impact on the retrieved cloud microphysical properties. For example, a sensitivity study conducted by Mace et al. [2002] showed that a difference of up to a factor of 4 in IWC can be caused by the ice crystal habit assumption for one particular cloud retrieval method. Protat et al. [2011] also showed using in situ microphysical observations in tropical cirrus that the use of density–diameter relationships for single habits does produce large biases relative to bulk IWC observations: from −50% for bullet rosettes to +80% for aggregates. Wang and Sassen [2002] indicated that different particle mass-length assumptions could change the Ze/IWC relationship by up to 50%. Therefore, the differences in ice crystal habit assumptions are partially responsible for the large discrepancies found between retrievals, such as the ice re differences between MICROBASE, MACE, DENG, and COMBRET shown in Figure 8. The exact impact of different ice crystal habit assumptions on each retrieval algorithm needs further sensitivity analysis, which is beyond the scope of the current study and will be done in the future.

[35] Finally we talk about the impacts of different particle size distribution (PSD) assumptions on cloud retrievals. A common assumption is that a single PSD is sufficient to determine the scattering properties of the ice crystals within the cirrus layer [Mace et al., 2002]. Gamma (or modified gamma) and exponential (a gamma PSD with order zero) PSD are the two widely used unimodal PSD assumptions for current ice cloud retrieval algorithms. However, bimodal distributions have been found for a large fraction of cirrus clouds [Mitchell et al., 1996; Mace et al., 2002; Y. Zhao et al., 2011]. Recent field measurement during the Small Particles In Cirrus (SPARTICUS) campaign at SGP suggests that the bimodal PSD is not always better than the unimodal PSD to fit the measured particle size distributions [Schwartz and Mace, 2011]. Therefore, it is not clear that any given PSD assumption is better than another in a generic sense. However, there is no doubt that different PSD assumptions impact retrieval results. For example, it is obvious that more particles are concentrated in the small size area for clouds with a lower order gamma PSD. Therefore, smaller re will be obtained for retrievals with a low order gamma PSD in comparison to the same retrievals but with a high order gamma PSD [Deng and Mace, 2006].

4. Statistical Analysis of Cloud Retrievals

[36] In this section, we use multiyear data between 2005 and 2008 at TWPC3 and between 2002 and 2007 at other sites to examine differences in the probability distribution functions for the cloud properties and show a statistical summary of the correlations and differences among the cloud retrieval products. The purpose of this analysis is to examine whether the differences found in section 3 are statistically robust. For each site, only the clouds for which all applicable retrieval products have valid values are considered.

4.1. Probability Distribution

[37] To better compare the cloud retrieval products, we classify the clouds into thin and thick clouds. Turner [2005] have shown that an AERI-based optimal estimation method is only valid for thin liquid clouds with optical depth (τl) less than 6 and Comstock et al. [2007] have used an optical depth (τi) of 0.3 to classify optical thin and thick ice clouds. Unfortunately, we do not have independent measurements of the cloud optical depths for all five ARM sites. Instead, cloud geometric depth (ΔH) determined from CMBE is used to classify thin and thick clouds. Note that the cloud base/top in CMBE is determined as the layer when CMBE cloud fraction becomes larger/smaller than 30% for a single layer cloud. ΔH can be related to the cloud optical depth through

display math
display math

where LWC and IWC are in g/m3 and ΔH is in m. The empirical equation for ice optical depth (equation (9)) is from the study by Heymsfield et al. [2003]. Considering typical values of liquid re = 8 μm, LWC = 0.1 g/m3 and ice IWC = 0.01 g/m3, the ΔH of 300 m and 600 m roughly correspond to optical depths of 6 for liquid and 0.3 for ice clouds, respectively. Therefore, we classify geometric thin and thick clouds according to whether ΔH less or greater than 300 m for boundary layer overcast clouds, and 600 m for high level ice clouds.

[38] Figure 10 shows the probability distributions of cloud LWC and liquid re from different cloud retrieval products for geometrically thin and thick boundary layer overcast clouds at 3 ARM fixed stations of SGP, NSA and TWPC3. Similarly, Figure 11 shows the statistical distributions of high level ice cloud properties. The numbers shown in the figures are the total cloud samples used for this statistical analysis at each site, and the colors represent the frequency of cloud samples that lie within different ranges of LWC, liquid re, IWC or ice re.

Figure 10.

Statistical distribution of cloud LWC and liquid re for geometric thin (ΔH ≤ 300 m) and geometric thick (ΔH > 300 m) boundary layer overcast clouds. The numbers in the title are the total cloud samples used for this statistical analysis at each site, and the colors indicates the frequency of cloud samples which lie within the specified range of LWC and liquid re.

Figure 11.

Statistical distribution of cloud IWC and ice re for geometric thin (ΔH ≤ 600 m) and geometric thick (ΔH > 600 m) high level ice clouds. The numbers in the title are the total cloud samples used for this statistical analysis at each site, and the colors indicates the frequency of cloud samples which lie within the specified range of IWC and ice re.

[39] For both geometrically thin and thick clouds, Figures 10 and 11 show similar comparison results as those found in Section 3. These similarities confirm that the large differences found between various retrieval products are not case dependent, but statistically robust. The PDFs shown here also give rough ranges of cloud microphysical properties for different cloud retrieval products. For boundary layer stratus, LWC and liquid re from most retrievals generally vary between 0–0.6 g m−3 and 3–13 μm, respectively. For high level ice clouds, IWC and ice re from most retrievals typically varies between 0–0.5 g m−3 and 10–70 μm, respectively. As a reference, observations from several major field campaigns [McFarquhar and Heymsfield, 1996; Lawson et al., 2001; Dong et al., 2002; Heymsfield et al., 2004; McFarquhar et al., 2007; Yost et al., 2011] show that most stratus LWC and liquid re measurements typically vary between 0.01–0.8 g m−3 and 3–20 μm, respectively. For cirrus clouds, the observations show most IWC generally in a range of 0.001–0.5 g m−3. The aircraft measurements also show a large amount of small ice particles, which are likely influenced by particle shattering in the process of measurement [McFarquhar et al., 2007; Protat et al., 2011]. Compared to these limited aircraft measurements, the cloud microphysical properties from most cloud products studied here lie within reasonable ranges statistically. Further evaluation of these cloud retrievals with a collection of specific aircraft measurements will be done in the future.

4.2. Statistical Summary

[40] Taylor diagrams [Taylor, 2001] are used to examine the statistical differences of cloud properties among these examined retrievals. Since there are no long-term aircraft observations available for all these examined sites, we use MICROBASE as a reference (the black point marked ‘M’ in Figure 12) in each Taylor diagram because of its availability for all conditions at all five sites. However, we should keep in mind that this does not mean MICROBASE is more accurate than others. The Taylor diagrams in Figure 12 provide correlations, centered root-mean square (RMS) differences, and ratios of temporal standard deviations (Sother/SMICROBASE, where S is temporal standard deviation) for cloud LWP, LWC, liquid re, IWP, IWC, and ice re between a given cloud retrieval and MICROBASE. The centered RMS difference between the other retrievals and MICROBASE is proportional to the distance to the point on the x axis identified as “M.” Since the centered RMS error is the error distance after the bias (or offset) between two data sets has been removed, we use different colors to discriminate the difference in mean cloud properties between a retrieval and MICROBASE. These colors indicate the ratios of mean values of the cloud properties from a specific cloud product compared to MICROBASE, Xi/XM, where X represents cloud properties, i is any specific cloud product and M is MICROBASE.

Figure 12.

Taylor diagrams show the statistical correlation coefficients, relative standard deviation (Sother/SMICROBASE, where S is standard deviation) and centered root-mean square errors for all retrieval products relative to MICROBASE regarding (a) LWP, (b) LWC, (c) liquid re, (d) IWP, (e) IWC, and (f) ice re. The numbers in the plots indicate the different cloud retrieval products. The colors indicate the different ratios of mean values of the cloud properties (X) from a specific cloud product compared to MICROBASE.

[41] For boundary layer overcast liquid clouds, the correlation between MICROBASE and other cloud retrieval products is generally low. Figure 12a shows that the average LWP is similar (within 40%) among most cloud products. The exceptions are WANG and COMBRET which give smaller values except at TWPC1. Figure 12b shows that the cloud LWC is significantly smaller in MACE and CLOUDNET and larger in WANG and COMBRET (TWPC2 and TWPC3) than MICROBASE compared to their uncertainties shown in Table 2. Considering that LWC has been scaled by the MWR LWP in all cloud products, the different inter-comparison results shown in Figures 12a and 12b indicate that different liquid cloud depths have been used in the cloud products. For example, at TWPC3, the averaged liquid cloud depths should be the largest for CLOUDNET, and the smallest for COMBRET. Figure 12b also shows smaller RMS difference for MACE and SHUPE_TURNER than for CLOUDNET and WANG respective to MICROBASE. This is because that LWC in MICROBASE, MACE and SHUPE_TURNER are derived using similar Ze-based algorithms while LWC in CLOUDNET and WANG are adiabatic estimates. Figure 12c shows that cloud liquid re in MICROBASE is systematically smaller than that in others, which is most likely due to its droplet number concentration assumption.

[42] In general, the ice cloud properties obtained from different cloud products have much higher correlation coefficients and smaller RMS differences than those for liquid clouds, particularly for IWC and IWP. However, this doesn't necessarily mean that the ice cloud retrievals are any more certain than liquid cloud retrievals. The commonality of radar data used by most ice cloud retrievals is likely the reason for the better correlations found in Figures 12d, 12e and 12f. There are high correlations and low RMS differences for IWC in MICROBASE, MACE, CLOUDNET, and SHUPE_TURNER, associated with their similar Ze-based IWC retrieval algorithms. However, the different parameters used in the algorithms make IWC in SHUPE_TURNER systematically smaller than that in MICROBASE. In contrast, COMBRET and VARCLOUD generally show significantly larger averaged values and RMS differences in IWC relative to those Ze-based cloud retrievals, which should be related to their combined radar and lidar basis. Ze-Vd based cloud retrievals (DENG and RADON) show significantly larger IWC at TWPC3 and similar (within 60%) IWC at other sites compared to Ze-based cloud retrievals. For ice re in high level ice clouds, the correlation coefficient between examined cloud products mainly lies between 0.5 and 0.8. Respective to MICROBASE, the correlation is relatively weaker for SHUPE_TURNER and MACE compared to others. The most likely reason is that the ice re of thin ice clouds in SHUPE_TURNER and MACE are derived using radiance based optimal estimation method while other cloud products are from radar (or radar-lidar) based retrievals. Figure 12f also shows that the ice re from temperature-based MICROBASE is significantly smaller than from radar-lidar and radar-only based cloud retrievals at low (3 TWP sites) and middle (SGP) latitudes, except for from RADON. For ice re from cloud products other than MICROBASE, COMBRET, DENG and VARCLOUD are similar (within 40%) and larger than RADON at TWPC3, DENG is significantly smaller than SHUPE_TURNER at NSA, and other cloud products are similar (within 40%) with each other at SGP, TWPC1 and TWPC2.

[43] The correlations of retrieved cloud properties among cloud products other than MICROBASE, which are not shown in Figure 12, are also generally low for liquid cloud properties and much higher for ice cloud properties. The potential reasons are similar as discussed above.

5. Summary and Discussions

[44] This study systematically documents the differences among nine cloud retrieval products (i.e., MICROBASE, MACE, CLOUDNET, DENG, SHUPE_TURNER, WANG, COMBRET, VARCLOUD and RADON) and explores the potential causes for these differences in term of retrieving the microphysical properties of boundary layer overcast clouds and high level ice clouds at the ARM SGP, NSA, and TWP sites. Following are the main findings.

[45] For boundary layer liquid clouds, clear differences in liquid re and LWC have been found among cloud products associated with differences in retrieval instrument basis and assumptions used in different retrieval techniques. Different vertical structure can result from their retrieval basis and parameters, such as the LWC from radar based methods with different empirical parameters versus the LWC from an adiabatic calculation. Clear differences in LWC shown in this study also indicate the significant role of cloud retrieval input and constraint parameters (e.g., cloud boundaries and MWR LWP).

[46] For high level ice clouds, higher correlations in re and IWC are found among the cloud products associated with their common use of Ze. The magnitude of the correlation coefficient is highly related to the similarity of their retrieval instrument basis (radar basis, radar-lidar basis, radar-T basis, and T basis). Similar to boundary layer liquid clouds, clear differences in ice re and IWC for high level ice clouds have been found among cloud retrieval products, which are associated with the differences in retrieval basis, parameters, and assumptions. Note that the differences of cloud properties caused by differences in underlying assumptions have not been explicitly examined in this study, particularly the different ice crystal habit assumptions.

[47] In summary, this study has shown the large systematic differences between various cloud products based on long-term statistical analysis. These differences are even often greater than those uncertainties prescribed in Table 2. This fact indicates that the estimated uncertainties of retrieval methods, as reported by different algorithm developers, are too optimistic. This study has also proposed possible reasons for these differences in term of their retrieval basis, assumptions, parameters, as well as the retrieval inputs and constraints. However, to quantify the effects from different factors and ultimately determine the best estimate of cloud properties under different conditions, further constraints of these retrievals with more observations and dedicated sensitivity analyses with different combinations of the retrieval factors are highly needed.

[48] A better understanding of the factors leading to differences in cloud properties between various cloud products will facilitate efforts to quantify cloud retrieval uncertainties and develop a best estimate of cloud microphysical properties with error bars. Developing a uniform input and constraint data file based on ARM value-added products, which provide a best estimate of these required fields, can help considerably reduce these differences as found in earlier studies [Dunn et al., 2010; Huang et al., 2011]. To address the uncertainty issue within current cloud retrievals, ARM is making an effort to assemble the ground based cloud retrievals into an ARM cloud retrieval ensemble data set (ACRED) [C. Zhao et al., 2011]. This product could provide a rough estimate of uncertainties in these cloud retrievals based on current instruments and retrieval techniques provided that the algorithms were reasonably designed to retrieve cloud properties for a certain type of clouds. One concern with the current ACRED is that the uncertainty in each of the ensemble members has not been determined for all meteorological conditions. To address this issue, one could generate an ensemble data set for each of the algorithms by perturbing key parameters and/or changing key assumptions used in these selected retrieval methods. This will help improve our understanding of the uncertainty associated with each of these retrieval methods and provide necessary information to further quantify the uncertainty using statistical methods such as the Bayesian approach. Another idea is to create observation system simulation experiment (OSSE) data sets and run the algorithms on these [McFarlane et al., 2002; Hogan et al., 2006b]. While the designs of forward models in OSSE limit the use of this method, a carefully constructed comparison might be able to determine which algorithm is more accurate under which conditions, and what the effects of different assumptions are. Moreover, the accuracy of assumptions and parameters in the cloud retrievals can be evaluated and tested with more in situ aircraft data and observed surface and top of atmosphere (TOA) radiative fluxes [Mlawer et al., 2008]. In addition, with the knowledge of the strengths and weakness of retrieval algorithms based on sensitivity studies, it is possible to determine the optimal technique for certain types of clouds under specific condition and then to develop a best estimated cloud properties data set by merging the optimal algorithms for all conditions together. In short, more research is needed to better understand and reduce the uncertainty in current cloud retrievals.


The notation section provides a description of the symbols and abbreviations in Table 3.


particle size distribution.

Exponential PSD

N(r) = N0 exp(−λr).

Lognormal PSD

math formula.

(modified) gammaPSD

math formula.

Normalized PSD

N(Deq) = N0*F(Deq/Dm).


normalized PSD.

N, N0

number concentration, number concentration intercept.


number concentration intercept proportional to IWC/Dm4.

r, r0

radius, modal radius.

λ, α



standard width of lognormal distribution.

Deq, Dm

‘equivalent melted’ diameter, volume weighted diameter.


discrete ordinate radiative transfer model [Stamnes et al., 1988].


line-by-line radiative transfer model [Clough et al., 1981, 1992].


moderate resolution atmospheric transmission version3 [Berk et al., 1989].

δ-2 stream model

δ-2 stream radiative transfer model [Toon et al., 1989].

γ, μ0

cloud transmissivity ratio, cosine of solar zenith angle.

T, P, I

temperature, pressure, and spectral radiation.


cloud base temperature.


liquid water path, rain rate.


water equivalent radar reflectivity.

Vd, σd

radar Doppler velocity, Doppler velocity spectral width.


lidar extinction coefficient.

Wm, Wσ

mean air vertical velocity, standard deviation of the vertical motion.


a function of ….

a, b



cloud ice fraction.

Zeliquid, Zeice

radar reflectivity from liquid contribution, ice contribution.


liquid water content, ice water content.

re, re_layer

effective radius, layer average effective radius.


optical depth.


empirical parameterization method.


radiation matching optimal estimation method.


forward approach which theoretically derives the cloud properties with forward models.


[49] This study is supported by the DOE ARM and ASR programs. Work at LLNL, PNNL and BNL was performed under the auspices of the DOE, Office of Science, Office of Biological and Environmental Research by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344, by Pacific Northwest National Laboratory under contract DE-AC05-76RL01830, and by Brookhaven National Laboratory under contract DE-AC02-98CH10886, respectively. The contribution of M. D. Shupe is performed under the support by DOE grant DE-FG02-05ER63965. The Cloudnet project was funded by the European Union from grant EVK2-2000-00065. The development of the VARCLOUD retrieval was funded by NERC grant NE/C519697/1. The contribution of G. G. Mace is performed under the support by DOE grant DE-FG0398ER62571. The contribution of D. D. Turner is performed under the support by DOE grant DE-FG02-06ER64167 as part of the ARM program. The contribution of Z. Wang is performed under the support by DOE grant DE-SC0006974 as part of the ASR program. The three reviewers of this paper are gratefully acknowledged for their constructive comments.