SEARCH

SEARCH BY CITATION

Keywords:

  • model evaluation;
  • radar forward operator;
  • CloudSat

Abstract

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Description of forward operator and global model
  5. 3. Uncertainty of radar reflectivity forward-model assumptions
  6. 4. Comparison of IFS and CloudSat radar reflectivity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

This article explores the uncertainties associated with evaluating a global atmospheric model with radar reflectivity observations. A forward operator for radar reflectivity (ZmVar) is described and used for the comparison of the ECMWF global numerical weather prediction model short-range forecasts with radar data from CloudSat. A sensitivity study is performed to determine which differences can be attributed to either specific radar forward operator assumptions or to deficiencies in the global model. The results show that model-derived reflectivities are particularly sensitive to the definition of subgrid precipitation fraction, as precipitation dominates the radar reflectivity signal, but also to the choice of particle size distribution and scattering properties of the different hydrometeor categories. However, there are a number of consistent differences in the reflectivity comparison that are significantly larger than can be explained by the sensitivity tests. This suggests that these discrepancies are due to deficiencies in the model cloud and precipitation frequency of occurrence and hydrometeor water contents. These include too frequent occurrence at high altitudes, too low occurrence in the Southern Hemisphere storm track and an overestimate of rain in warm-phase low cloud. The study shows the value of CloudSat for evaluating the model in terms of radar reflectivity and highlights the importance of taking into account forward operator uncertainties for both model evaluation and data assimilation applications. Copyright © 2012 Royal Meteorological Society


1. Introduction

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Description of forward operator and global model
  5. 3. Uncertainty of radar reflectivity forward-model assumptions
  6. 4. Comparison of IFS and CloudSat radar reflectivity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

The representation of cloud is still one of the most uncertain aspects of global climate and numerical weather prediction (NWP) models, with direct and indirect impacts on the hydrological cycle, radiation and atmospheric dynamics. Information on the global distribution of cloud and precipitation vertical profiles from active radar data has the potential to highlight the deficiencies and compensating errors in these models, with benefits for operational weather forecasting, seasonal forecasting and climate simulation. In particular, the cloud radar on board the CloudSat satellite (Stephens et al., 2002) has provided an extensive and detailed description of the vertical distribution of clouds and precipitation across the globe since 2006. Additionally, a number of ground-based radars have provided multiyear time series of radar reflectivity profiles, including research radars in Europe (Illingworth et al., 2007) and the Atmospheric Radiation Measurement (ARM) programme with fixed and mobile facilities in the Tropics, midlatitudes and high-latitudes (Ackerman and Stokes, 2003).

A prerequisite for comparing the model fields with observed data for model evaluation or data assimilation is an appropriate transform, either from model to observation space, or from observation to model space, in order to provide a fair comparison. In the first case, a forward operator must be available to process model output in terms of the radar observable (i.e. equivalent radar reflectivity factor). This is the approach followed, for example, by Bodas-Salcedo et al. (2008), Marchand et al. (2009), Zhang et al. (2010), and by Wilkinson et al. (2008) for lidar observations. In the second approach, the observations are transformed to model variables, often using a priori information or combining with additional data from other observing instruments, for example the CloudSat Level-2 products of cloud mask, cloud phase and water contents or the combined radar–lidar algorithm described by Delanoë and Hogan (2010). This is the method followed for the evaluation of ice water content by Waliser et al. (2009), Wu et al. (2009) and Delanoë et al. (2011). Whichever approach is followed, there are many assumptions that need to be made in either transforming the model variables to the observed quantity or estimating the model variable from the observations. Both methods need to take account of not only the different parameters observed and prognosed by the model, but also the different spatial scales in the model and observations, and uncertainties and error characteristics of the observations. There is certainly benefit in investigating the two approaches separately in order to reduce the chance of misinterpreting results and to provide confidence in the conclusions of the model evaluation required for further model parametrization development and improvement.

The focus of this article is the model to observation approach applying a radar reflectivity forward operator to GCM (general circulation model) cloud and precipitation fields and directly comparing with temporally and spatially matched observations. There is some uncertainty whether the source of any reflectivity difference stems from the amount of condensate prognosed by the model, from the representation of subgrid-scale variability (e.g. cloud and precipitation fraction) or from the microphysical assumptions that are used for the forward model. If these ambiguities are not understood or dealt with appropriately, the model evaluation can be misleading. Previous work in the literature evaluating model radar reflectivity has mentioned uncertainties, but as a secondary issue (e.g. Bodas-Salcedo et al., 2008). The purpose of this article is to show that an appropriate treatment of uncertainty should be a necessary part of the evaluation process to determine the origin of model error. The article does not attempt to produce a complete error analysis of all possible uncertainties, but uses examples of typical uncertainties in microphysical and subgrid variability assumptions to place the emphasis on understanding some of the limitations of the comparison. We hypothesize that such an analysis can provide enough information to separate deficiencies in the model prediction of hydrometeor occurrence and water contents from subgrid and microphysical assumptions in the forward operator for radar reflectivity. Such a study is also a preliminary step in assessing errors for assimilation of radar reflectivity data in an NWP model.

The ECMWF global numerical weather prediction model is evaluated using the 94 GHz radar reflectivity observations from the CPR instrument on board the CloudSat satellite (Stephens et al., 2002). Section 2 describes the relevant details of the model, CloudSat data and the radar reflectivity forward operator (ZmVar) used in this study. Section 3 investigates some of the possible uncertainties in assumptions in the forward operator including hydrometeor particle size distributions, particle scattering properties, and subgrid cloud and precipitation fraction. The sensitivity study describes impacts on the radar reflectivity with implications for the comparison of model and observations. Section 4 describes the results of the comparison of the ECMWF model-derived radar reflectivity with CloudSat data for the month of January 2007 and highlights which signals are associated with uncertainties in the forward operator assumptions and which are likely to be associated with discrepancies in model forecast cloud and precipitation occurrence and hydrometeor contents. Section 5 provides a discussion of the results, conclusions and future directions.

2. Description of forward operator and global model

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Description of forward operator and global model
  5. 3. Uncertainty of radar reflectivity forward-model assumptions
  6. 4. Comparison of IFS and CloudSat radar reflectivity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

2.1. Integrated Forecast System global model and CloudSat data

At ECMWF, an operational global numerical weather prediction model (called the Integrated Forecast System or IFS) is used across a range of resolutions for medium-range NWP forecasts, an ensemble forecast system and lower resolution monthly forecasts, seasonal forecasts and reanalysis. The data used in this study were obtained using the 36R4 version of the model (operationally used from November 2010 to May 2011) with a spectral truncation of T511 (approximate grid spacing of 40 km) and 91 levels in the vertical up to 80 km altitude. Cloud and precipitation processes are described by prognostic equations for cloud liquid, cloud ice, subgrid cloud fraction, stratiform rain and snow with a diagnostic parametrization for subgrid precipitation fraction. This cloud scheme is based on Tiedtke (1993) but has since been extended and modified significantly. This model version introduces additional prognostic variables for precipitating rain and snow and also partitions the cloud condensate into separately prognosed cloud liquid and ice variables for an improved representation of the mixed phase. Microphysical processes are parametrized for nucleation, phase transitions (e.g. condensation, evaporation) and collection processes (e.g. accretion, aggregation) to transfer water between the different categories. Although the fraction of a grid box that is covered by cloud (the subgrid cloud fraction) is prognosed in the model with sources and sinks from different processes, the precipitation fraction is a diagnostic variable based on the cloud profile with maximum overlap in the vertical. An additional assumption of subgrid heterogeneity in the precipitation leads to a linear reduction of the precipitation fraction during the rain and snow evaporation process (i.e. a proportion of the precipitation coverage is assumed to have lighter precipitation that evaporates completely), but the formulation is empirical and uncertain. The convection parametrization in the IFS is linked to the cloud scheme as a source of cloud mass and fraction from convective detrainment (Tiedtke, 1991). However, convective rain and snow profiles are diagnosed from the scheme without the explicit prediction of a precipitation fraction, although a value of 5% is used to calculate the local subgrid precipitation rate for the evaporation term. In-cloud inhomogeneity of cloud properties within a grid box is not considered at this time.

The CloudSat data used in this study are from the 2B-GEOPROF product (Marchand et al., 2008). As a screening criterion, a CloudSat vertical bin is considered to be hydrometeor-filled when the GEOPROF cloud mask product has values between 20 and 40.

In order to compare the model data with CloudSat, the IFS model profiles are temporally and spatially matched to the CloudSat satellite track using the following procedure. The IFS model is initialized at 12 UTC every day, and three-hourly output from forecast hours 12 to 36 are concatenated together to provide a continuous record of model data. Therefore, model data are always within 1.5 h or less of the observation time. Only the model grid points nearest to CloudSat observations in space and time are extracted from the global model forecast fields to form a ‘curtain’ of model profile data along the CloudSat track. To avoid issues with orography, only cases over ocean are considered. Model output quantities include temperature, humidity, cloud and precipitation contents (liquid and ice phase), thus providing the necessary input for the radar reflectivity forward operator.

2.2. The ZmVar radar reflectivity forward operator

At ECMWF, a radar forward operator was required in feasibility studies on the assimilation of ground-based (35 GHz) radar observations and the Precipitation Radar (13.5 GHz) on board the Tropical Rainfall Measuring Mission satellite (Benedetti and Janisková, 2004; Janisková, 2004; Benedetti et al., 2005; Lopez et al., 2006). In that context, a forward model was developed to simulate reflectivities from the atmospheric conditions prescribed by the IFS (ZmVar, reflectivity model for variational assimilation). The ZmVar was originally designed to meet the requirements of the assimilation system, i.e., to allow the coding of its adjoint counterpart, to be computationally efficient and to have a certain degree of flexibility (e.g. for the definition of the optical properties). For this study ZmVar was extended to simulate observations from CloudSat, carrying a 94 GHz nadir-looking radar with a sensitivity of approximately −30 dBZ.

Figure 1 schematically describes how reflectivities are computed by ZmVar. The inputs to ZmVar are vertical profiles of the model cloud and thermodynamic variables coincident with the observations (Step 1). For computational efficiency, ZmVar uses a pre-calculated table of hydrometeor optical properties (extinction and backscattering coefficients; Step 2). This look-up table prescribes equivalent radar reflectivity factor (Zh, called reflectivity hereafter) and volumetric extinction (βh) at a given wavelength (λ) of predefined hydrometeor types (h) across a range of temperatures and hydrometeor contents (w). Currently six hydrometeor types are modelled within ZmVar, consistent with the categories in the ECMWF model: cloud liquid, cloud ice, stratiform rain, convective rain, stratiform snow and convective snow.

thumbnail image

Figure 1. Schematic diagram of ZmVar.

Download figure to PowerPoint

Values for Zh and βh (for the specified temperature and hydrometeor content) for hydrometeor type h are generated off-line integrating the backscattering cross-section equation image and the extinction cross-section equation image over all particle sizes with diameter D, from an assumed particle size distribution (PSD), Nh(D) (see section 2.4 for PSD details):

  • equation image(2.1a)
  • equation image(2.1b)

where equation image and equation image are predefined values of limits for the particle size. Variable Kw is defined as:

  • equation image(2.2)

with mw being the water refractive index at the radar frequency and a reference temperature of 273 K. To ensure the same calibration format of CloudSat, ZmVar uses a value of 0.75 for |Kw|2.

A value of reflectivity Zh due to each single hydrometeor type h is evaluated by means of a bilinear interpolation on the values of temperature and hydrometeor content of the input profile (Step 3). Radar reflectivities of all hydrometeor types Z are then combined:

  • equation image(2.3)

where NHY D is the number of hydrometeor types. The attenuation of the signal along the radar path due to absorption by hydrometeors and atmospheric gases must be taken into account as attenuation can be significant at 94 GHz (Step 4). Attenuated reflectivities Za are calculated using:

  • equation image(2.4)

where τ is the total optical depth (for gases and clouds) between the considered height and the top of the atmosphere (Step 5). In ZmVar the attenuation due to gases is evaluated using the model of Liebe (1985) and Liebe et al. (1992). The final value of reflectivity is obtained taking into account the profiles of cloud and precipitation fraction prescribed for the atmospheric state (Step 6, see section 2.3 for details). It is worth noting that the modelling of radar multiple scattering is currently not included in ZmVar although it can be important at the CloudSat frequency for the most intense clouds (e.g. Battaglia et al., 2010).

So far, the ZmVar forward model closely resembles the approach used elsewhere, including the QuickBeam radar forward operator (Haynes et al., 2007), which is part of the CFMIP (Cloud Feedback Model Intercomparison Project) Observation Simulator Package (COSP, Bodas-Salcedo et al., 2011). The following section discusses the steps outlined above in detail and will emphasize where ZmVar diverges from other existing radar forward models.

2.3. Treatment of cloud and precipitation fraction

The ZmVar is designed to simulate radar reflectivity from an atmospheric model with horizontal resolutions coarse enough to require some representation of subgrid cloud and precipitation variability. Even when the model provides explicit information on subgrid cloud and precipitation fractions, the overlap in the vertical within a grid column must be taken into account. In this section, two alternative treatments of cloud/precipitation fraction implemented in ZmVar for the IFS are described. The first one consists of a single-column treatment that was part of the (original) version of ZmVar used in previous studies (e.g. Benedetti et al., 2005). The second more sophisticated approach follows the multicolumn concept recently implemented in ZmVar. Both are described here in order to highlight the benefit of the multicolumn approach used in this study compared with the previous single-column approach.

2.3.1. Single-column approach

In the single-column approach, subgrid cloud and precipitation fraction are taken into account by the following steps:

  • 1
    For each layer l, the IFS specifies the cloud fraction equation image only for cloud liquid and cloud ice. The ZmVar assigns a precipitation fraction equation image to rain and snow starting from equation image, making the assumption of random overlap for cloud and maximum overlap for precipitating hydrometeors.
  • 2
    The in-cloud value (denoted by′) of hydrometeor content equation image in layer l for hydrometeor type h is evaluated from the box-average values given by the model, scaled according to the corresponding subgrid fraction:
    • equation image(2.5)
    where equation image is equation image for cloud liquid and cloud ice, or equation image for rain and snow.
  • (3)
    The in-cloud reflectivity equation image and extinction equation image are then evaluated using equation image, as previously described by Eqs (2.1a) and (2.1b). Grid box averages Zl and βl are then obtained by ‘scaling’ the in-cloud values equation image and equation image according to the cloud and precipitation fractions and summing on the total number of hydrometeor types NHY D:
    • equation image(2.6a)
    • equation image(2.6b)
    Note that the values of equation image are very similar to those corresponding to the (un-scaled) equation image because for all hydrometeor types the extinction coefficient has an almost linear dependence on the hydrometeor content. Note also that reflectivities are expressed in linear units (i.e. mm6m−3).
  • 4
    Finally, the reflectivity for direct comparison with the radar, Za, is evaluated taking into account the attenuation along the path:
    • equation image(2.7a)
    where equation image is the average optical depth between the lth layer and the top of the atmosphere defined as:
    • equation image(2.7b)
    and Δhi is the depth of the ith layer. Note that levels are numbered from the top of the atmosphere and so l = 1 indicates the top layer. Also, the attenuated reflectivity for each layer in Eq. (2.7a) is calculated based on the total extinction to the layer centre.

With this approach, no information is needed about the way clouds vertically overlap within the box. Its main advantage is in its computational efficiency. However, this is achieved at the expenses of accuracy because it neglects the (strong) nonlinearity in the relationship between radar reflectivity and cloud/precipitation amount.

2.3.2. Multicolumn approach

An alternative method that avoids the intrinsic inaccuracy of the previous treatment of cloud/precipitation fraction is the multicolumn approach, as used in the Subgrid Cloud Overlap Profile Sampler (SCOPS), which is part of the International Satellite Cloud Climatology Project (ISCCP) simulator (Webb et al., 2001) and COSP (Bodas-Salcedo et al., 2011). This method uses a pseudo-random sampling process to generate an ensemble of subgrid cloud profiles representing the distribution within the model grid box. It takes the input vertical profiles of cloud fraction to generate a specified number of horizontally homogeneous cloud profiles. The SCOPS algorithm splits each grid column into a number of subcolumns NCOL in which each layer is either completely filled or completely free of cloud. The cloud cover in each vertical layer and subcolumn is therefore either 0 or 1. The SCOPS version used here assumes maximum random overlap for cloud (Geleyn and Hollingsworth, 1979; Tian and Curry, 1999) but maximum overlap for precipitation. The precipitation overlap scheme is discussed in more detail in section 3.1.

For the multicolumn case, the attenuated reflectivity for layer l averaged across all subcolumns equation image can be written as:

  • equation image(2.8a)

where

  • equation image(2.8b)

and equation image (either equation image for cloud or equation image for precipitation) is either 0 or 1 in the lth layer of the kth subcolumn. Note that, apart from numerical issues, the single-column and multicolumn approximations produce the same grid-box average unattenuated reflectivity. Differences are found only for the attenuated reflectivity Za due to the treatment of optical depth, with the multicolumn case providing a more realistic representation of overlap for the reflectivity profile. The ideal number of columns NCOL for the multicolumn approach needs to be chosen to be sufficiently accurate but not too computationally expensive. The accuracy of the IFS cloud fraction forecasts is certainly not expected to exceed 1%, hence for a reference case we assume that 100 subcolumns are sufficient to reproduce a grid-point's cloud fraction profile with adequate precision, and to resolve the variability introduced to the simulated radar signal due to cloud overlap. Figure 2 shows the mean and standard deviation of the absolute difference of reflectivity between various numbers of subcolumns and the reference 100 subcolumns for a day of global data. In each model grid box, reflectivity is averaged over all hydrometeor-filled subcolumns. The results show the mean absolute difference and standard deviation reduces more slowly for 20 subcolumns or more. When choosing the number of subcolumns, additional consideration should be given to the value of the convective precipitation fraction. If the fixed convective precipitation fraction of 5% in the IFS model cannot be represented exactly on subcolumns, it is effectively rounded. As a default, 20 subcolumns are used in ZmVar as a reasonable compromise between accuracy and computational cost, and that represents the 5% fraction exactly by one subcolumn.

thumbnail image

Figure 2. Mean and standard deviation of the absolute difference between ZmVar reflectivity obtained using the number of subcolumns in the abscissa and a reference of 100.

Download figure to PowerPoint

2.4. Microphysical assumptions in the forward operator

As mentioned in section 2.2, ZmVar generates radar reflectivities using a pre-calculated table containing volumetric extinction and equivalent reflectivity for the relevant frequency (94 GHz in this case) and for predefined hydrometeor types. For the generation of this look-up table, definitions of hydrometeor single particle scattering properties and PSDs are required. If possible, these assumptions should be consistent with the ones made in the forecast model used to generate the cloud profiles input to the simulator. However, there are two reasons why independent assumptions may be required. Firstly a model may not prescribe consistent assumptions across the parametrizations in the model (clouds, convection, radiation) or even within a parametrization. Secondly, reflectivity is particularly sensitive to the large end of the particle size distribution (D6 in Rayleigh regime), whereas microphysical processes depend more on the mass-weighted part of the PSD (D3) and (scattering of) radiation on the particle cross-sectional area (D2). Therefore the emphasis on the PSD may be different for the radar forward operator than for the model physics. In addition, there is significant spatial and temporal variability in the atmosphere of hydrometeor PSDs and scattering properties and significant uncertainty in representing this information appropriately in a global model. Hence the approach here is to make independent assumptions for PSDs and particle optical properties in the forward operator. The impacts of uncertainties in these assumptions will be assessed in the next section. Given below are the details of the parametrizations for the PSDs and single particle scattering properties chosen for each hydrometeor category.

The PSD for rain in ZmVar is modelled using the ‘normalized’ gamma distribution of Illingworth and Blackman (2002). For cloud liquid, a log-normal PSD is used, which, unlike the exponential, does not change shape with changes in the water content. The standard set-up in ZmVar uses the values of median diameter and logarithm width for maritime clouds as reported in table 3 of Miles et al. (2000). For cloud ice, the size distribution is modelled as a gamma distribution, which has a spread wider than the log-normal function. Again, the shape of the gamma PSD does not depend on the amount of ice content. For snow, the PSD parametrization developed by Field et al. (2007) is used. Based on in situ observations, it includes an increase in the relative number of larger particles at warmer temperatures, representing the effect of particle aggregation on the shape of the distribution. In their study, two distinct parametrizations are proposed for midlatitude and tropical clouds. In our implementation the midlatitude PSD function is used since it is representative of a wider range of meteorological conditions. For both rain and snow, the same PSD has been used to model stratiform and convective precipitation.

Ice particles can assume a variety of shapes depending on their growth environment and on past evolution. Sphericity is often used as a convenient approximation, but it is not a valid assumption in the majority of cases. A more realistic approach is the use of typically observed ice particle habits (shapes). This implies a more complex (and computationally expensive) solution to the problem of finding the corresponding optical properties. The approach used here, as demonstrated in several articles, is the use of the discrete dipole approximation (e.g. Evans and Stephens, 1995). For cloud ice, the values of single particle backscattering and extinction given by Liu (2008) for five-bullet rosettes (Type 07) are used. This choice is justified by the fact that in situ observations reveal that bullet rosettes are the most commonly observed shapes in the upper layers of ice clouds (Heymsfield et al., 2002). The single particle properties of both stratiform and convective snow have been modelled using the results for aggregates of columns evaluated in Hong (2007).

To fully determine the PSD of frozen particles for a given total mass, the particle density also needs to be specified. However, this cannot be done arbitrarily as it must be consistent with the choice of particle shape. Under the assumption of random orientation and based on the particle projected area and volume, the actual density can be expressed as a power law, function of the diameter:

  • equation image(2.9)

where D is interpreted as the maximum dimension of the particle.

The PSD functional form, fixed parameters, particle shape and density for each hydrometeor type are summarized in Table 1 for the standard configuration of ZmVar. Following Korolev et al. (2000), who found that most particle size distributions are dominated by irregular shapes, an ‘aggregate’ model is used for snow. The density of Brown and Francis (1995) is employed for these particles. Westbrook et al. (2007) found that this relationship is in good agreement with the density values derived using their particle modelling of aggregation. Note that for rain and snow, no distinction is made in the table between the stratiform and the convective type since they are currently modelled in the same way.

Table 1. Definitions of particle size distribution (PSD), particle shapes and densities used in the generation of the standard configuration of the ZmVar look-up table.
HydrometeorPSDPSD parametersParticle shapeParticle density ρ(D) = αDβ ρ: g cm−3, D: cm
Rainequation imageμ = 5,Sphere (Mie)α = 1.0
  NL = 0.08 cm−4 β = 0.0
SnowField et al., 2007 Aggregates (DDA,α = 5.615e-3
   Hong et al., 2007)β = -1.1
Cloud liquidequation imageDg=14 μm,Sphere (Mie)α = 1.0
  σg = 0.3 β = 0.0
Cloud iceequation imageμ = 2, Dn = 50 µmSix-bullet rosettesα = 0.0077
   (DDA, Liu, 2008)β = -0.88

The relationships between hydrometeor content and radar reflectivity contained in the look-up table are shown by the curves in Figure 3. For cloud droplets, non-precipitating ice, rain and solid precipitation (snow) hydrometeor types, two lines are plotted, referring to the two extremes of the temperature-dependent relationships. The left one refers to the minimum temperature and the right one to the maximum temperature specified in the look-up table. It is worth mentioning that the water permittivity model of Liebe et al. (1991) has been used for temperatures as low as −20° C, a value for which this model was not tested. This was a necessary choice because, at present, a model for the permittivity of supercooled cloud drops is not available.

thumbnail image

Figure 3. Mapping of hydrometeor content into reflectivity using ZmVar. Lines show relationships between mass content of each hydrometeor type and the corresponding simulated reflectivity as prescribed in the look-up table. Colour shading shows the relative occurrence of the pairs' total hydrometeor content vs. reflectivity obtained running ZmVar on a month of model forecast data (January 2007). (a) Subcolumn data for total hydrometeor content and unattenuated reflectivities. (b) Subcolumn data for total hydrometeor content and attenuated reflectivities. (c) Model grid-box averaged total hydrometeor contents are plotted against grid-box averaged attenuated reflectivities. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Download figure to PowerPoint

In the same figure, the colour shading indicates the relative occurrence of samples in total hydrometeor content vs. reflectivity space, obtained from ZmVar for a dataset of IFS model profiles from January 2007, extracted using the co-locating procedure with CloudSat described in section 2.1. In Figure 3(a) the in-cloud total hydrometeor contents are compared with the corresponding unattenuated reflectivities evaluated for the single subcolumns. Data points lying along the lines correspond to situations where reflectivity is the result of a dominant hydrometeor type. This condition occurs more often for rain, and for snow when close to the freezing level (due to large snow particles dominating the backscatter). This also occurs for ice cloud at colder temperatures where it is the only type of hydrometeor. All points away from the individual hydrometeor bands/lines are the combined effect of more than one hydrometeor type evaluated on subcolumns. The central panel of Figure 3 shows a similar figure, but using the attenuated reflectivities. As expected, the attenuation shifts high reflectivity samples towards lower reflectivity values, particularly for rain and snow, reducing the maximum reflectivity from around 20 dBZ to 10 dBZ. Since our analysis is performed on quantities at the model grid resolution, the right panel of Figure 3 presents the relative occurrence of grid-box mean (attenuated) reflectivities vs. grid-box mean total hydrometeor contents. The pattern is much smoother in this case due to the averaging of cloud/precipitation, and the initial reflectivity vs. content relationships are therefore less apparent.

As an example, Figure 4 shows a cloud structure observed by CloudSat and represented by the IFS using ZmVar. CloudSat observations at the original resolution and averaged (horizontally and vertically) to the model grid resolution are shown in panels (a) and (b), respectively. The corresponding model data with simulated reflectivities from ZmVar are shown in panel (c). Simulated observations are evaluated at model resolution by averaging the reflectivities of the hydrometeor-filled subcolumns in the grid box, considering only values above the minimum detectable threshold (−30 dBZ for CloudSat). Comparing panels (b) and (c), we note that the forecast model is able to realistically reproduce the structure of this event, and the general distribution and magnitude of radar reflectivity values. The contingency mask is given in Figure 4(d) showing the correspondence between model and observations. Note that the difference in the lowest kilometre is due to the removal of those CloudSat data contaminated by surface clutter. The grid-box hydrometeor fractions for CloudSat and the model (panels (e) and (f), respectively) show that the model broadly captures the differences between the higher cloud fraction region in the frontal system between 7000 and 8000 km (although with some underestimate) and the lower cloud fraction in the convective region between 4700 and 5500 km.

thumbnail image

Figure 4. (a) Reflectivity of a cloud system as observed by CloudSat on 2 January 2007; (b) the same CloudSat reflectivities gridded and averaged to the IFS model resolution; (c) the corresponding simulated reflectivities using ZmVar on the ECMWF model short-term forecasts. (d) The contingency mask between observed and simulated reflectivities. (e,f) The hydrometeor fraction in each grid box for the observations and model respectively. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Download figure to PowerPoint

3. Uncertainty of radar reflectivity forward-model assumptions

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Description of forward operator and global model
  5. 3. Uncertainty of radar reflectivity forward-model assumptions
  6. 4. Comparison of IFS and CloudSat radar reflectivity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

As discussed in sections 2.3 and 2.4, a number of assumptions about particle properties and subgrid cloud and precipitation fraction are required in the process of modelling radar observations. The choices made may be generally valid, but have significant variability and uncertainty and therefore they cannot be considered the most appropriate in every situation.

In this section, sensitivity tests are performed, changing the treatment of precipitation fraction and microphysical assumptions in ZmVar. While it would be desirable to perform a rigorous error analysis that relates perturbations in initial parameters in a quantitative manner to the forward model output errors, some of the uncertainties in the ZmVar output stem from parametrization choices (such as the overlap scheme and choice of ice habits) that do not lend themselves to a perturbed-parameter type error analysis. The sensitivity tests performed here, however, will help in understanding the impact of uncertainties in ZmVar on the simulated reflectivities and the attribution of the sources of model error.

3.1. Sensitivity to subgrid precipitation fraction and radar sensitivity threshold

Rain and snow precipitation make a significant contribution to, and often dominate, the radar reflectivity signal due to the larger size of the precipitating hydrometeors. While the vertical overlap of clouds is routinely considered for radiative transfer calculations in models, less attention is given to precipitation fraction and how precipitation and clouds are arranged in the vertical. In the IFS, a precipitation fraction is used to convert grid-box average rain and snow contents into the equivalent of an in-cloud (or rather, in-precipitation, or local) value for the evaporation process. The strategy for determining precipitation overlap with ZmVar is illustrated in Figure 5. For a given model cloud fraction (panel a), cloud overlap is calculated first (as described in section 2.3.2) providing a set of subcolumns with binary cloud cover (panel b). The cloudy subcolumns are ranked according to the number of cloud-filled levels in each subcolumn (panel c). Panel (d) shows the stratiform precipitation fraction, as obtained from the IFS. Subcolumns are then flagged as precipitation-filled based on the existence of large-scale precipitation at a given model level, and the existence of either cloud at the current level, or precipitation in the level above (panel e). Thus precipitation fraction is directly related to cloud formation and is maximally overlapped with cloud (referred to here as the ‘PMAX’ precipitation fraction)

thumbnail image

Figure 5. Schematic diagram describing the precipitation overlap scheme. (a) Cloud fraction as provided by the model. (b) Cloud overlap is determined and expressed as a binary cloud fraction on subcolumns. (c) Subcolumns are ranked by number of cloud-filled vertical levels. (d) Large-scale precipitation fraction is provided by the model, as well as a flag for the existence of convective precipitation (black bar). (e) Precipitation and clouds are maximally overlapped as a first guess (PMAX). (f) As an optional step, the first guess precipitation fraction is adjusted to match the model precipitation fraction from (d) (PEVAP). If vertical levels are flagged as containing convective precipitation, a fixed fraction of subcolumns (one in this example) is flagged as containing convective, as well as large-scale precipitation

Download figure to PowerPoint

As an optional step, the resulting first-guess stratiform precipitation fraction on subcolumns is adjusted to be consistent with the precipitation fraction parametrization of the IFS (panel f). At a given level, precipitation is set to zero first in the least cloudy subcolumns if the fraction has to be reduced, and is added to the most cloudy subcolumn if it needs to be increased. Overall, the adjustment leads to a reduction of the first-guess precipitation fraction in ZmVar. This may be interpreted as reintroducing a reduction in precipitation fraction due to re-evaporation, which is not explicitly accounted for in ZmVar, but is considered in the IFS (referred to here as the ‘PEVAP’ precipitation fraction). As already mentioned, IFS does not predict a fraction estimate for convective precipitation, but does assume a fixed fraction of 5% for the evaporation of convective precipitation. If convective precipitation exists on a vertical level (marked by a black bar in panel (d), the number of subcolumns corresponding to 5% are flagged, again starting with the most cloudy subcolumn (panel f). If stratiform precipitation already exists, the total cloud fraction in the model column is not altered. Where no stratiform precipitation previously existed, the convective precipitation fraction is added. The convective and stratiform precipitation properties may be treated differently in ZmVar, but in the version used here the same properties are chosen for both.

An important model parameter to validate is the reflectivity occurrence (Oz), i.e the fraction of data points (either model subcolumns or CloudSat profiles) with reflectivities above a certain threshold. The reflectivity occurrence is sensitive to the treatment of subgrid-scale precipitation fraction and overlap. The observed reflectivity occurrence over ocean from CloudSat for a month-long period in January 2007 is reproduced in Figure 6 (a) (grey solid line). The model's cloud fraction is shown as a dashed curve. This corresponds to the scenario where the cloud fraction directly from the model with no forward operator applied is compared with a radar-derived total hydrometeor mask (cloud and precipitation). As expected, neglecting the contribution of precipitation to reflectivity occurrence leads to a strong underestimate of the occurrence in the mid- to lower troposphere, while occurrence is overestimated in the upper troposphere, where small ice particles would probably produce reflectivities below the detectability threshold. This type of comparison is clearly misleading and highlights the importance of using a forward operator on both cloud and precipitation.

thumbnail image

Figure 6. (a) Global mean reflectivity occurrence (Oz) for ocean points during the month of January 2007, based on various precipitation fraction estimates. CloudSat observations shown as solid grey line. Model cloud fraction only is shown as dashed line (precipitation fraction is neglected). Using ZmVar first-guess precipitation fraction (PMAX) and applying a −30 dBZ threshold on the resulting simulated reflectivities produces the dash-dotted line. The mean reflectivity occurrence when the model's precipitation fraction (PEVAP) is used is shown as the dotted line. (b) Sensitivity of hydrometeor fraction to dBZ threshold applied to simulated reflectivity. CloudSat observed hydrometeor fraction shown in grey.

Download figure to PowerPoint

The sensitivity of reflectivity occurrence when applying the forward operator to both cloud and precipitation for two different assumptions of precipitation fraction is also presented in Figure 6(a). Applying a −30 dBZ threshold to the simulated reflectivities using the PMAX option for precipitation overlap (Figure 5(e)) results in much greater values (dash-dotted line) than for the cloud alone. The alternative precipitation fraction parametrization, including consideration of precipitation re-evaporation (PEVAP, Figure 5(f)) produces slightly lower occurrence (dotted line) than for PMAX, but both are equally close to the observed CloudSat profile.

A value of −30 dBZ is chosen for the detection threshold of the simulated radar reflectivity as this is close to the sensitivity of the CloudSat radar. However, panel (b) of Figure 6 illustrates the sensitivity of the model's reflectivity occurrence to this threshold value. Shown in grey is the observed reflectivity as in Figure 6(a). The solid black line shows the corresponding values from the model with the PEVAP precipitation fraction and applying the −30 dBZ threshold. Lowering this value to −35 dBZ slightly increases the occurrence (dashed line), while raising it to −25 dBZ results in fewer subcolumns exceeding the threshold (dotted line). The impact in both cases is most pronounced above approximately 5 km, where ice and precipitating snow dominate. As illustrated in Figure 3, there is a maximum in the occurrence of reflectivities from the cloud ice close to the −30dBZ reflectivity range, which explains the higher sensitivity at altitudes above the freezing level. Figure 6 also shows the higher uncertainty due to the precipitation fraction parametrization than to the radar sensitivity threshold.

3.2. Sensitivity to the representation of hydrometeor properties

Modelling the backscattering and extinction properties of frozen hydrometeors is crucial for properly simulating the radar signal. A number of assumptions need to be made about the PSD and/or particle shape for the forward modelling computations. The IFS model does not provide explicit parameters for the radar reflectivity forward operator as there are currently a number of different assumptions used for different microphysical processes and, as for many models, a lack of consistency between the cloud, precipitation, convection and radiation parametrizations. Recently, Kulie et al. (2010) and Hiley et al. (2011) proposed a methodology to bound the uncertainties in the forward modelling of microwave brightness temperatures and in the snowfall retrieval from radar observations. Following a similar approach, the microphysical assumptions made in ZmVar are replaced with possible alternatives in order to evaluate the impact on the synthetic reflectivities.

3.2.1. Alternative microphysical assumptions

The default microphysical assumptions used by ZmVar as described in the previous section represents one possible set of choices, but there are many uncertainties in these assumptions. In the sensitivity study described below, the basic idea driving our choice of ‘perturbations' to assess the impacts of uncertainties on the model-observation comparison is to use alternatives that we believe to be realistic rather than extreme.

  • 1
    Rain PSD. For rain, the drop size spectrum is variable and dependent on precipitation rate and regime. In the standard set-up the Marshall–Palmer particle size distribution is used, because this is similar to the mean value found, for example, by Tokay and Short (1996). We have investigated the sensitivity to this assumption replacing it with a different PSD, still with a constant-intercept exponential function, but with a larger value (0.3 cm−4 instead of 0.08 cm−4). This change corresponds (for a given water content) to a reduction of the number of larger particles and is representative of lighter precipitation events.
  • 2
    Cloud Liquid PSD. The sensitivity of cloud liquid to the PSD assumption has been investigated by decreasing the mean radius of the original PSD defined in ZmVar from 7 µm to 4 µm. As reported in Miles et al. (2000), in situ measurements suggest that continental clouds typically have smaller drops, and the new PSD would be more representative over land.
  • 3
    Snow PSD. The importance of the PSD for snow has been investigated replacing the Field et al. (2007) midlatitudes' PSD used in the standard set-up with the one proposed by the same authors for the Tropics. This PSD is still temperature-dependent and, as expected, prescribes a higher number of particles with larger sizes.
  • 4
    Ice particle shape and density. Modelling of optical properties of frozen particles is not straightforward because it is difficult to define a shape that is generally valid. While this is an issue for both precipitating snow and non-precipitating ice particles, we focused our test on the latter (cloud ice). A test has been performed replacing the optical properties corresponding to the shape/density used in the original table (five-bullet rosettes) with the ones of hexagonal columns from Liu (2008).

The new relationships of hydrometeor content versus reflectivity (dashed lines) are compared with the original relationships (solid lines) in Figure 7 (left panel). As in Figure 3, there are two lines per hydrometeor, each representing the minimum and maximum temperature specified in the table. The changed PSD for rain leads to a noticeable decrease in reflectivity for lower contents (up to 4 dBZ) while, due to the Mie effect, there is a small increase for contents above 0.1 g m−3. The new cloud liquid PSD leads to reflectivities only just above the CloudSat sensitivity threshold. The new snow PSD results overall in larger reflectivities and extinctions. The new habit for cloud ice brings an increase in reflectivity of about 1 dBZ. Figure 7 (right panel) shows the contribution to extinction due to each hydrometeor. We note that rain and cloud liquid extinctions are always higher than for frozen hydrometeors, by up to one order of magnitude. However, snow extinction for the largest amounts and warmest temperature can be as high as that of cloud liquid.

thumbnail image

Figure 7. Relationships between mass content of each hydrometeor type and corresponding simulated (a) reflectivity and (b) volumetric extinction as prescribed in the ZmVar look-up table. For each hydrometeor type, the two lines correspond to the minimum and maximum value of temperature range used. Solid lines are for the standard configuration, dashed ones correspond to the changed set-up. Envelopes of shaded-colour lines show the temperature dependence (from −70° C (most left) to 0°C (most right), with a 1°C stepping) of snow size distribution in the standard configuration, while the black-dashed line refers to the changed case, temperature-independent. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Download figure to PowerPoint

3.2.2. Results of sensitivity tests

To understand how the changes to the microphysical assumptions translate into changes of the simulated reflectivities, a set of experiments were performed with ZmVar for data from January 2007. The changes to microphysical assumptions described above were organized into three new look-up tables, each based on the original reference version with the following alterations: (i) modified cloud liquid PSD, (ii) modified snow PSD, and (iii) modified rain PSD and cloud ice particle shape combined.

The first quantity we are interested in is the simulated reflectivity, averaged (in linear space) over each grid box considering those subcolumns with reflectivity exceeding the sensitivity threshold of −30 dBZ. The analysis of the results is performed considering zonal averages over 3° wide latitude bands.

The second quantity used for the study is the reflectivity occurrence. This is evaluated as the fraction of the profile subcolumns falling within each latitude band and having a reflectivity above the detection level. This definition is independent of the grid-box size resolution and, as discussed in section 4, it avoids any direct comparison of occurrence on single grid points where the uncertainty on the representativity of observations in terms of fraction/occurrence can be an issue.

Figures 8(a) and 9(a) respectively show the zonal means of the reflectivity occurrence and reflectivity (when present) obtained with the reference version of ZmVar with the standard look-up table. We note that the largest reflectivities occur where rain is present or where there is a significant amount of snow (particularly in the Tropics), highlighting the critical role that precipitation fraction has on the reflectivity occurrence.

thumbnail image

Figure 8. Latitudinal mean of reflectivity occurrences with reflectivity above the −30 dBZ CloudSat sensitivity threshold. (a) Simulated using the reference set-up in ZmVar. (b) to (d) The difference with respect to the reference, changing the (b) size distribution for snow, (c) size distribution for cloud liquid and (d) rain size distribution and cloud ice optical properties. Note the different scales for each panel. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Download figure to PowerPoint

thumbnail image

Figure 9. Same as Figure 8, but showing the latitudinal mean of reflectivity. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Download figure to PowerPoint

The impact of the microphysical assumption changes on the calculated reflectivities from ZmVar are also shown in Figures 8 and 9. Panel (b) shows the differences for the snow PSD change, Tropics versus midlatitude PSD assumptions from Field et al. (2007). The Tropics snow PSD results in larger reflectivities for the smallest water content, which leads to an increase in the mean occurrence (panel b of Figure 8). As expected, mean reflectivities are larger for all altitudes at midlatitudes (panel b of Figure 9). However, the analysis of the pattern in the tropical regions is more complex. The tropical PSD gives larger mean reflectivity in the Tropics in a band centred at about 5 km and extending for 2–3 km. The opposite occurs in a narrow band just above the freezing level, where the reflectivities of the modified snow PSD are lower by less than 1 dBZ. In fact, as shown in Figure 7, at the warmer temperatures, if the water content is larger than 10−2 g m−3, the new PSD prescribes lower snow reflectivity. Again in the Tropics a more evident region of reduction (up to 2 dBZ) in mean reflectivity is present above 8 km. This is the effect of the large water contents of convective snow (not shown), which can exceed 1 g m−3. For these values, at colder temperatures the new PSD also gives lower reflectivities and larger extinctions than the original formulation.

On the third row of Figures 8 and 9 (panel c) is shown the effect of changing the PSD for cloud liquid. Since in this case a given cloud liquid content corresponds to a smaller reflectivity, we note a smaller occurrence (Figure 8). Less intuitively, the average reflectivities are marginally higher (∼ 1 dBZ). The reason for this is the effect of extinction on total attenuation due to cloud liquid water droplets, which is almost as strong as for rain (see Figure 7b), despite significantly weaker backscatter. This is a consequence of the fact that volumetric extinction is less dominated by the largest particles (there is a D3 dependence instead of D6 for reflectivity), and so although cloud water droplets are much smaller than raindrops, their effect on extinction can be important. The decrease in cloud liquid attenuation (due to the changed PSD) overwhelms the very small decrease in the cloud contribution to the total backscatter and overall produces larger reflectivities in the layers where both cloud liquid and rain are present.

The results of the modified rain PSD and cloud ice particle shape are combined in the bottom row of Figures 8 and 9 (panel d) as they affect distinctly different altitude bands. Both changes lead to a decrease in the values of simulated reflectivity. Figure 8 shows that the rain PSD change does not correspond to a significant decrease in the number of cases with reflectivity above the −30 dBZ threshold, because even very small amounts of rain (10−4 g m−3) can produce a detectable signal. However, the new rain PSD leads to a negative difference (up to 2 dBZ) evident between 40°S and 40°N below 5 km in Figure 9.

Changing the backscatter/extinction properties of cloud ice from bullet rosettes to hexagonal columns produces both a lower number of observations above the detectability threshold and smaller mean reflectivities (of about 0.5 dBZ). This decreases because the largest contribution to the reflectivity is given by particles with diameters smaller than 0.02 cm, and for these sizes the column shape is a less efficient scatterer than the bullet rosettes. As expected, the effect is noticeable only in the upper portions of the cloud, i.e. at heights where only cloud ice is present. Regions where cloud ice coexists with snow, even if present in larger amounts, show little or no difference as snow dominates the total reflectivity whenever it is present.

The result for other choices of ice particle shape indicates that there is significant sensitivity in the reflectivity at this altitude band. For example, choosing ‘hexagonal plates' produces larger differences (many dBZ) of opposite sign (not shown). With only two categories of ice in the model (‘cloud ice’ and ‘snow’) it is difficult to represent the many types of ice cloud particles observed in the atmosphere. However, as mentioned earlier, the choice of bullet rosettes in the reference version of ZmVar is based on the high occurrences of this type of particle from aircraft observations.

4. Comparison of IFS and CloudSat radar reflectivity

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Description of forward operator and global model
  5. 3. Uncertainty of radar reflectivity forward-model assumptions
  6. 4. Comparison of IFS and CloudSat radar reflectivity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

In order to assess the impacts of the microphysical and precipitation fraction changes in ZmVar on the radar reflectivity evaluation, the simulated reflectivities from the ECMWF IFS model are compared against coincident CloudSat measurements for the month of January 2007, as described in section 2.1.

4.1. Comparison methodology

The error introduced by comparing a ‘curtain’ of observations with the volume of a model grid box is unavoidable in the absence of cross-track observations. In addition, inhomogeneity of cloud properties within a grid box is currently neglected in the ECMWF model. The multicolumn approach adds some variability due to cloud overlap, but cloud properties on each model level are identical for all subcolumns. As a consequence, the output of the ZmVar forward model is inherently unable to capture the full variability of radar reflectivities observed by CloudSat within a grid box. Thus it would be misleading to directly compare simulated reflectivity on subcolumns to observed individual radar profiles and expect comparable variability within the area of a grid box.

For a fair comparison, judicious averaging to a spatial scale that the model can resolve, e.g. the grid-box scale or greater, can alleviate the problem of unresolved cloud inhomogeneity. This is the approach chosen for the following discussion. The ‘in-cloud’ properties shown in Figures 9 and 10 are derived by first calculating the grid-point average in-cloud properties from all cloudy subcolumns/radar profiles within a grid column, then subsequently averaging the grid-point values for each 3° latitude belt.

thumbnail image

Figure 10. (a) Latitudinal mean of reflectivity occurrence obtained from CloudSat observations with reflectivity above −30 dBZ for January 2007. (b) to (e) The difference between ZmVar/IFS and CloudSat (model minus observations) for different assumptions in the modelling of hydrometeor scattering properties: (b) using the standard configuration; (c) changing snow size distribution; (d) changing cloud liquid size distribution; (e) changing rain size distribution and cloud ice optical properties. (f) Differences between ZmVar/IFS and CloudSat in mean fraction when precipitation fraction is calculated using the cloud fraction alone and maximum precipitation overlap (PMAX) rather than the reduced precipitation fraction (PEVAP) used in the reference ZmVar. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Download figure to PowerPoint

However, averaging introduces additional uncertainty. The relatively small number of observations falling into the grid-box area (approximately 36 radar profiles for the 40 km grid resolution model used here) make it inherently more likely that either clear or overcast conditions will be observed. Hedging towards clear/overcast conditions may be reduced by considering a larger number of observations along the track when comparing with a model grid box. Two alternative averaging methods have been explored to arrive at a latitude-band mean value. In the first, individual cloudy radar profiles/model subcolumns are averaged within the latitude band, avoiding any reference to the model grid. Note that with this approach, the ‘in-cloud’ properties are those of each individual profile/model column, rather than the properties of a model column (and correspondingly averaged radar profiles). As a second alternative, the radar profiles are averaged along a longer section of the track, corresponding to three model grid boxes. Both alternative approaches to averaging produce results (not shown) only slightly different from those in Figures 9 and 10, thus not affecting any of the conclusions drawn from the sensitivity tests.

4.2. Impact of changes on reflectivity occurrence

The zonally averaged CloudSat reflectivity occurrence for the month of January 2007 is given in panel (a) of Figure 10. In the same figure, panel (b) shows the difference in reflectivity occurence between ZmVar (defined using a −30 dBZ detectability threshold) and CloudSat observations. Due to ground clutter in the radar signal in the lowest range gates, the data are removed in the lowest kilometre. A small but systematic overestimation of reflectivity occurrence exists at higher altitudes, while an underestimation dominates below in the lower troposphere. The height where the model bias changes sign is 7–8 km in the Tropics and Southern (summer) Hemisphere, decreasing to 3 km in the Northern (winter) Hemisphere. The amount by which the occurrence is overestimated is quite small, usually below 0.05, but the largest differences (up to 0.1) occur in the Tropics, above 7–8 km. The underestimation of reflectivity occurrence tends to be more pronounced at lower levels in the extratropics with differences up to 0.15.

The effects on the intercomparison of the changes to the microphysical assumptions discussed in section 3 (Figures 8 and 9) are presented in panels (c, d, e) of Figure 10. When compared with the reference, the qualitative pattern of differences is similar for all cases. The only noticeable change is the further increase in the overestimation in the tropical upper troposphere from the change of the snow PSD (panel c). Panel (f) presents the effect of using the PMAX precipitation fraction rather than PEVAP. As shown in Figure 6, this change results in significantly larger precipitation fractions. The regions of underestimations are replaced by a small overestimation in the extratropics and more significant overestimation in the Tropics, with the exception of the summer hemisphere high latitudes where the low bias is only slightly reduced.

4.3. Impact of changes on radar reflectivity

Figure 11(a) shows, again for the month of January 2007, zonal means of CloudSat reflectivities. Panel (b) shows the difference between the corresponding averages of IFS simulated reflectivities and the CloudSat observations using the reference ZmVar configuration. From this figure, the differences can be categorized into four main areas:

  • 1
    larger model reflectivities (up to 10 dBZ) at lower altitudes consistent across all latitudes but most pronounced in the Tropics;
  • 2
    lower model reflectivities above an altitude of 7–8 km in the Tropics;
  • 3
    larger model reflectivities (few dBZ) at mid- and high-latitudes that extend for 2–3 km near the cloud top;
  • 4
    reasonably good agreement at mid- and high latitudes above the freezing level in the midtroposphere, in the regions of precipitating snow, but a small underestimate in reflectivity in the Southern (summer) Hemisphere.
thumbnail image

Figure 11. Same as Figure 10, but showing the latitudinal mean of reflectivity. This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Download figure to PowerPoint

The impact of the sensitivity experiments on the average simulated-minus-observed reflectivity differences is shown in the lower four panels of Figure 11. Consistently with Figure 9, panel (c) shows that the change to the snow PSD increases the reflectivity slightly in the midtroposphere in the extratropics, but the change is small compared with the magnitude of the differences between model and observations. Figure 11(d) shows that changing the PSD of cloud liquid also does not significantly affect the intercomparison because the impact is small. The change to the rain PSD with smaller particle sizes, representative of lighter precipitation, has a more significant effect on the comparison (Figure 11(e)), reducing the large positive bias below the freezing level. In the same panel, we note that using the single particle properties of a different ice particle shape does not affect the results significantly, although there is a small reduction in the bias of about 0.5 dBZ in the upper troposphere in the extratropics and a small increase in the Tropics. As noted in section 3, an additional sensitivity experiment using the ‘hexagonal plates' shape instead increases the reflectivity, which leads to a higher discrepancy with observations in the extratropics (not shown). Observational evidence shows the existence of plates is confined to a temperature range between −10° C and −20° C (Yoshida et al., 2010) and is therefore of limited applicability across a wider temperature range.

Finally, Figure 11(f) presents the effect of using the PMAX precipitation fraction instead of PEVAP. As shown in Figure 6(a), the PMAX assumption increases the hydrometeor fraction below 8 km altitude, which leads to lower ‘in-cloud’ precipitation amounts and lower reflectivities, and a reduction of the positive biases present in the lower troposphere.

The results have focused on the mean differences between model and observed radar reflectivities in order to provide a zonal mean view of the first-order model deficiencies. However, the probability density function (PDF) of reflectivity and how this varies with height (or temperature) can provide additional information. Shown in Figure 12 are the diagrams for the global CloudSat data for January 2007 at native and model grid resolution as well as the simulated reflectivity from the reference version of the model. The contour plots result from the PDFs of reflectivity at each temperature range (individually normalized). We note how the averaging process only has a small impact on the observed PDF with only a slight reduction of reflectivities. However, the comparison with the model PDF highlights some of the main differences seen in the zonal cross-sections, including: (i) a wider range of reflectivities at very cold temperatures (< 220 K) associated with the general overestimate in midlatitudes and underestimate in the Tropics for high-altitude ice cloud; (ii) the narrower distribution and underestimate of mean reflectivities closer to midlevels (220 K to 270 K); and (iii) the overoccurrence of higher reflectivites (0 to 5 dBZ) at temperatures above freezing associated with small precipitation size particles, with a corresponding underestimate of reflectivities associated with cloud size drops (−30 to −25 dBZ).

thumbnail image

Figure 12. Frequency distribution of radar reflectivity with temperature. (a) CloudSat observations for January 2007. (b) The same data after averaging at model resolution. (c) The ZmVar simulated reflectivities (model grid box mean). Solid line indicates the mean reflectivity value. Dashed line indicates the fractional occurrence of reflectivity (above the detection threshold). This figure is available in colour online at wileyonlinelibrary.com/journal/qj

Download figure to PowerPoint

4.4. Discussion

The purpose of performing the sensitivity experiments was to estimate the impact of uncertainties in some of the forward operator cloud and precipitation assumptions on the IFS model radar reflectivity comparison with observations. This is a necessary step in order to determine which differences can be confidently attributed to deficiencies in the representation of cloud and precipitation hydrometeor contents and occurrences in the model. The comparison between the IFS model and CloudSat data together with the results of the sensitivity study in Figures 10–12 does highlight a number of differences that are likely to be due to IFS model deficiencies. These include too frequent hydrometeor occurrence at high altitudes, too low occurrence in the Southern Hemisphere storm track and an overestimate of reflectivity in warm-phase low cloud probaly due to overoccurrence of rain. Each of these deficiencies is discussed in turn below.

  • 1.
    Upper tropospheric ice cloud. The reflectivity in the upper troposphere (> 7 km altitude) is dominated by ice clouds with relatively small ice water content (i.e. cirrus), and is affected by uncertainties in the microphysical assumptions for the ice cloud variable in ZmVar. However, the consistent signal suggests the overoccurrence of reflectivity in Figure 10(b) could be due to excessive cloud cover in the model, or cloud ice water contents that are too high. It is known that CloudSat misses a significant amount of thin cirrus as it is below the radar sensitivity threshold (Mace et al., 2009; Delanoë and Hogan, 2010). This is taken into account in the model data by applying a similar radar sensitivity threshold in ZmVar, but this fact does mean that an overestimate in ice cloud content would bring more cloud over the minimum threshold and result in an apparent overestimate in ice cloud occurrence. The consistent overestimate of reflectivity in the extratropics in Figure 11(b) supports this hypothesis, but in contrast, the low reflectivity bias in the tropical upper troposphere suggests that the reasons are more complex. In the presence of precipitating ice, even the smallest amounts correspond to radar signals above the sensitivity threshold. In this case the overestimation of reflectivity occurrence shown in Figure 10(b) for the tropical upper troposphere would lead to lower reflectivity (similarly to what is shown in Figure 11(f)). Another possible reason leading to a negative bias could be the lack of a specific parametrization of convective hydrometeors, but this requires further investigation.
  • 2.
    Low-level precipitating cloud. The overestimate of reflectivity in the lower troposphere across all latitudes, but most prominent at lower latitudes, suggests too frequent occurrence of rain, leading to reflectivities that are much too high compared with the equivalent reflectivity for non-precipitating cloud (as seen in Figure 12). This is consistent with other studies that show many GCMs, including the IFS, tend to overestimate the occurrence of light rain (Stephens et al., 2010). There is also an issue in representing the PSD for light rain shown by the rain PSD sensitivity experiment in Figure 11, where the reflectivity bias was reduced at low altitudes when a modified PSD more appropriate for light rain was used. Observations show that the PSD for rain can vary significantly for a given rain water content depending on the meteorological situation, whereas the Marshall-Palmer PSD representation typically used in GCMs is based on observations from heavier rain events. Improvements to the parametrizations of production, evaporation and representation of PSDs for the light rain mode (drizzle) in models are clearly needed.
  • 3.
    Southern Hemisphere storm track. The frequency of hydrometeor occurrence is consistently too low in the Southern Hemisphere storm track between 50°S and 80°S in the lower and midtroposphere (Figure 10), even when the precipitation fraction is increased. This suggests a lack of cloud, which is consistent with other model comparisons with independent cloud and radiation observations and is a common problem in GCMs (Trenberth and Fasullo, 2010). The midtropospheric reflectivities are too low, suggesting the water contents of ice/snow are also underestimated. The causes of this discrepancy in models again require further investigation.

It is also worth pointing out the importance of representing subgrid precipitation fraction for the model evaluation against radar reflectivity, because the backscatter is dominated by large precipitating particles when present. The results for the sensitivity experiment, reducing the precipitation fraction with evaporation, show a change in sign for the occurrence bias in the mid- to low-troposphere (Figure 10f). Whether the precipitation fraction is diagnosed by the model, or represented solely in the forward operator, there is significant uncertainty in this parameter and clear sensitivity to the hydrometeor occurrence and average reflectivity due to the rain and snow, which will affect the interpretation of the model evaluation. There are also implications for parametrization as the local precipitation water contents (precipitation water content divided by the precipitation fraction) is the relevant quantity for calculations of sedimentation, evaporation, and collection, as well as being of potential importance for subgrid precipitation NWP products and hydrological applications, and further research is required to reduce uncertainties.

5. Conclusions

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Description of forward operator and global model
  5. 3. Uncertainty of radar reflectivity forward-model assumptions
  6. 4. Comparison of IFS and CloudSat radar reflectivity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

There is a clear need to understand the limitations and uncertainties in both the observations and any forward operator assumptions required for an evaluation of GCM cloud and precipitation fields. It is a necessary step in determining the origin of model error and suggesting the direction of model developments for improved prediction in NWP and climate. The aim of this article is to explore the uncertainties associated with evaluating a global atmospheric model with radar reflectivity observations and to attribute differences in the comparison to either uncertainties in the forward operator, or deficiencies in the model forecast cloud and precipitation fields. A forward operator for radar reflectivity (ZmVar) is described and used for the comparison of the ECMWF global numerical weather prediction model short-range forecasts with data from CloudSat radar. These observations provide a unique opportunity to evaluate the vertical profiles of cloud and precipitation globally and to help identify the source of model error that can often only be inferred from passive instruments. Overall, the short-range forecasts from the ECMWF model with the reference version of ZmVar produces a reflectivity field that is similar to the observations in terms of spatial distribution and magnitude (as shown by the example in Figure 2). Given this baseline, we are interested in improving the model further and therefore need to evaluate the remaining discrepancies compared with observations. A small set of experiments modifying uncertain aspects of the forward operator microphysical and subgrid precipitation fraction assumptions are used to assess the sensitivity of the model–observation comparison. The results show that model-derived reflectivities and hydrometeor occurrence are particularly sensitive to the definition of subgrid precipitation fraction, as precipitation dominates the radar reflectivity signal, but also to the choice of particle size distributions and scattering properties of the different hydrometeor categories.

Although the sensitivity study is by no means comprehensive, the choice of alternative assumptions gives a first estimate of the latitudes, altitudes and magnitudes of possible uncertainties in the radar reflectivity evaluation. The study shows that there are a number of consistent differences in the comparison that are significantly larger than can be explained by the forward operator uncertainties. This suggests that these differences are due to other deficiencies in the model forecast cloud and precipitation in terms of frequency of occurrence and hydrometeor water contents. These include too frequent hydrometeor occurrence at high altitudes, too low occurrence in the Southern Hemisphere storm track and an overestimate of rain in warm-phase low cloud. Although the underestimate of cloud in the Southern Hemisphere has been inferred previously from an evaluation of top-of-atmosphere radiation measurements, and the overoccurrence of drizzle in low cloud has been observed from land observations, CloudSat provides quantitative information over land and ocean and new information to determine the characteristics of model error relating to cloud and precipitation.

There are a number of limitations of the current study and possible avenues for further work. As mentioned earlier, a limited yet manageable set of sensitivity tests were performed to highlight the main uncertainties in the forward operator. A wider and more comprehensive quantitative error analysis was beyond the scope of this article but would certainly be beneficial for further understanding of the influence of different uncertainties in the model evaluation process, and crucial for future data assimilation of radar reflectivity observations.

The approach taken here was to separate the assumptions in the forward operator from assumptions in the model. The reason for prescribing microphysical assumptions was twofold. First, the ECMWF model currently does not prescribe consistent assumptions across the parametrized processes and second, the PSD assumptions to capture the small particle mode (D2, for radiation) and the dominant mass-weighted particle mode (D3, for microphysics), may not adequately represent the large particle weighted mode (D6) for radar reflectivity. However, we should certainly strive towards consistent assumptions across the model parametrizations where possible.

The representativity error due to the spatial sampling of the narrow 1.7 km swath of the CloudSat radar compared with the much larger grid-box area of the model has been addressed in only a simple way, by using a longer track of observations for comparison with model grid-point equivalents. This is an additional error that could be taken into account in the future (e.g. following the approach of Stiller, 2010), but preliminary work at ECMWF has shown this error to be small compared with the magnitude of typical errors between model and observations.

The study shows the value of CloudSat for evaluating the ECMWF model in terms of radar reflectivity, but also highlights the importance of taking into account forward operator uncertainties. Although the results are from a comparison of one particular NWP model and observations from CloudSat, the sensitivity results for the forward operator would apply equally to other models and to the use of data from ground-based radar observations (such as the ARM sites), although attenuation due to precipitation may pose a greater problem. The results of the article thus have general implications for GCM evaluation with radar reflectivity from ground-based or space-based radars.

Having determined the robust signals in the ECMWF model–CloudSat evaluation, the next step is to analyse the results in more detail for different seasons, specific regions and meteorological regimes, and identify possible causes of the model discrepancies to further improve the prediction of cloud and precipitation. An in-depth interpretation of the evaluation results and improvement of the model parametrizations will be the subject of a future investigation.

Acknowledgements

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Description of forward operator and global model
  5. 3. Uncertainty of radar reflectivity forward-model assumptions
  6. 4. Comparison of IFS and CloudSat radar reflectivity
  7. 5. Conclusions
  8. Acknowledgements
  9. References

This work was supported by the European Space Agency (ESA), Noordwijk, The Netherlands, under ESTEC Contract No. 21613/08/NL/CB and the DoE Atmospheric Radiation Measurement programme through an ECMWF fellowship. The NASA CloudSat Project is kindly acknowledged for providing the CloudSat data. The authors are grateful for the valuable comments and suggestions from three reviewers.

References

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Description of forward operator and global model
  5. 3. Uncertainty of radar reflectivity forward-model assumptions
  6. 4. Comparison of IFS and CloudSat radar reflectivity
  7. 5. Conclusions
  8. Acknowledgements
  9. References