Geophysical Research Letters

Statistical evaluation of MPA-RT high-resolution precipitation estimates from satellite platforms over the central and eastern Mediterranean



[1] A statistical verification of satellite precipitation estimates has been conducted for a one year period over the Central and Eastern Mediterranean. The NASA real-time Multi-satellite Precipitation Analysis (MPA-RT) data are verified against 73 raingauge data. The verification aims to assess the skill of these satellite estimates to detect rainy areas and to give information on the accumulated precipitation errors. The results show almost unbiased results for the low and medium precipitation thresholds, especially during the wet period of the year. At higher accumulation thresholds, the satellite data overestimate the rain events compared to the raingauges, especially during the dry period of the year, when the major part of precipitation is produced by isolated thunderstorms. Moreover the analysis showed that for the high precipitation amounts and during the whole period the probability of detection is quite low and the false alarm ratio is high (reaching ∼85% during the dry period).

1. Introduction

[2] Precipitation plays a fundamental role in the global water cycle and in forcing the large-scale dynamics of the general circulation of the atmosphere. Satellite observing platforms seem to offer the best possibility for accurately estimating the mean climatological distribution and variability of global precipitation, and several years of daily and monthly precipitation estimates are distributed by large scientific institutes. In addition, there have been recent efforts to provide precipitation estimates at better spatial and temporal resolution. Indeed, in the frame of the Tropical Rainfall Measurement Mission (TRMM) project, the National Aeronautic and Space Administration (NASA) has distributed gridded 3-h precipitation rate estimates with relatively high horizontal resolution (0.25 × 0.25 deg) since January 2002. These precipitation estimates are based on microwave information provided by various low-orbiting satellites, merged with infrared-based estimates from geostationary meteorological satellites [Huffman et al., 2003]. It is obvious that the provision of these data is very important, not only over the Tropics, but also over large water bodies in the mid-latitudes where conventional in-situ precipitation measurements are not easily available. The Mediterranean Sea is an area where significant precipitation activity occurs (especially during autumn and winter), but except over the surrounding land masses, in-situ measurements over the sea are extremely sparse or non-existent.

[3] The potential of using real-time precipitation estimates at high temporal and spatial resolution is wide: better monitoring of rainfall over the Mediterranean for climatological and hydrological applications, assimilation of the data to regional modelling systems (adding valuable mesoscale information on the initial stages of forecasts), verification of precipitation model forecasts, etc. The verification of the satellite precipitation estimates is key to establishing the usability of these products. The objective of this paper is to make a one-year verification of the new MPA-RT data sets of satellite precipitation estimates against the available raingauge measurements over the Central and Eastern Mediterranean. This verification will be qualitative (skill of the satellite estimates to detect the rainy events) as well as quantitative (quantity bias and absolute errors of precipitation accumulations estimates). Since this product is relatively new, there are not yet any relevant references in the literature. The Australian Meteorological Service is however performing statistical evaluation of this product over Australia and the results are available on a dedicated web page, and the NOAA Climate Prediction Center has a parallel effort for the continental U.S. (see

[4] The methodology is presented in Section 2, while Section 3 presents the results of the statistical evaluation of the precipitation estimates. Finally, Section 4 summarises the work and gives prospects for the future use of these data.

2. Methodology

[5] The precipitation estimates used in the frame of this study are provided by two different sets of sensors. First, microwave data are collected by various low-orbit satellites, including TRMM Microwave Imager (TMI), and the Special Sensor Microwave/Imager (SSM/I) on-board the Defense Meteorological Satellite Program (DMSP) satellites. All the available microwave data are converted to precipitation estimates by applying the Goddard Profiling Algorithm (GPROF) [Kummerow et al., 1996].

[6] The microwave data are used together with infrared data collected by the international constellation of geostationary satellites (METEOSAT for the Mediterranean region) in a probability-matched method [Huffman et al., 2003]. A near-real-time final product that only depends on microwave and infrared data is computed a few hours after the acquisition of data from the orbiting platforms. The acquisition and application of algorithms are performed at the Goddard Space Flight Center/National Aeronautics and Space Administration (GSFC/NASA hereafter). Additional information concerning the algorithms applied to satellite data to derive precipitation rate estimates is given by Huffman et al. [2003].

[7] The final dataset (precipitation rate) consists of gridded files with 0.25 × 0.25 deg horizontal resolution (3B42RT products), within the global latitude belt 60°S to 60°N. The data are made available in near-real time. The temporal resolution is 3 hours and the files of precipitation rate estimates are generated on synoptic observations times (00 UTC, 03 UTC,…, 21 UTC).

[8] In this study, the MPA-RT satellite precipitation estimates are validated against data from 73 raingauge stations located in the central and south-eastern part of the Mediterranean, covering the area 10°–34° E and 35°–45° N (Figure 1). Most of these raingauges are self-siphoning gauges capturing rain into a glass tube without automatic digital recording. The verification period spans the 12 months June 2002 to May 2003. The comparison is made for the precipitation accumulated on 12-h intervals. It should be noted that the winter 2002–2003 was very wet over the Eastern Mediterranean, permitting a test of the satellite estimates against many rainy events with very high accumulations of rain.

Figure 1.

Location of the 73 surface stations used for verification.

[9] For the verification procedure, the satellite estimate grid boxes whose centers fall within a circle of 0.25 deg radius around the raingauge site are averaged, weighted by the inverse of their squared distance from the raingauge. For the qualitative verification (namely detection of rainy events at different precipitation thresholds) the Frequency Bias Index (FBI), Probability of Detection (POD) and False Alarms Ratio (FAR) scores have been evaluated, based on a 2 × 2 contingency table (a: satellite yes, observation yes, b: satellite yes, observation no, c: satellite no, observation yes and d: satellite no, observation no). Using this notation for a, b, and c, FBI is written as:

equation image

[10] The FBI (or bias score) is a measure of the relative frequency of the estimated compared to the observed rain events and does not provide information on how well the estimation corresponds to the observations. FBI can indicate whether there is a tendency to underestimate (FBI < 1) or overestimate (FBI > 1) rainy events. It ranges from 0 to infinity and the perfect score is equal to 1.

[11] The FAR is:

equation image

[12] The false alarm ratio is equal to the number of false alarms divided by the total number of times that rain was estimated and it gives a simple proportional measure of the satellite's tendency to estimate rain where none was observed. It ranges from 0 to 1 and the perfect score is equal to 0.

[13] The POD is:

equation image

[14] The probability of detection is equal to the number of hits divided by the total number of observations of rain events. Thus it gives a simple proportion of observed rain events successfully estimated by the satellites. It ranges from 0 to 1 and the perfect score is equal to 1.

[15] The aforementioned statistical scores have been calculated for various precipitation thresholds: 0.1, 0.5, 1, 2, 5, 10 and 20 mm, in order to evaluate the skill of the estimates for light and heavy rainfall events.

[16] The quantitative verification is based on the calculation of the quantity bias (QB) and the Mean Absolute Error (MAE), following the formulas:

equation image
equation image

where N is the total number of pairs of satellite estimates (SATi) and raingauge accumulations (OBSi). The number of pairs of data for each range of precipitation was: 5144 for the 0.1–2.5 mm range, 1348 for the 2.5–5 mm range, 1596 for the 5–10 mm range, 1095 for the 10–20 mm range, 476 for the 20–40 mm range and 120 for the >40 mm range.

[17] The statistical verification was conducted for the whole one-year verification period and also for two sub-periods of 6 months each: the dry period including April, May, June, July, August and September and the wet period for the remaining 6 months. This subdivision will give further insight to the skill of the precipitation estimates during autumn and winter, when large-scale systems are affecting the area, as well as during spring and summer when the major part of precipitation in the area is related to thunderstorm activity.

3. Statistical Evaluation of Satellite Estimates

[18] Figure 2 shows the verification results for FBI (Figure 2a), FAR (Figure 2b) and POD (Figure 2c), for the whole period as well as for the dry and wet periods of the year. Inspection of the statistical scores for the 12-month period shows that both the satellite and the raingauges detect almost the same frequency of rain event as the FBI is close to the perfect score for the thresholds up to 10 mm (Figure 2a). For higher thresholds (20 mm) there is an overestimation of the frequency of rain events by the satellite. The FAR shows the best scores up to the 2 mm thresholds (Figure 2b), while there is a clear linear increase at all thresholds greater than 5 mm. Consistently, the POD decreases almost linearly with increasing precipitation thresholds (Figure 2c).

Figure 2.

(a) FBI calculated for eight precipitation thresholds (in mm) (b) as in (a) but for FAR, (c) as in (a) but for POD.

[19] During the wet period, the frequency of rain events is underestimated by the satellites since FBI is slightly smaller than 1 for all rain thresholds. During the dry period of the year, FBI for all thresholds shows a systematic overestimation of the frequency of rain events, especially in the highest thresholds. An abrupt increase to very high FBI is evident for thresholds greater than 2 mm. The FAR during the dry period consistently provides worse results than the wet period, reaching a value of 0.85 for precipitation thresholds greater than 20 mm (Figure 2b). Accordingly, POD is higher for the dry period since during this period of the year the satellite estimates give a large number of rain events, increasing thus the number of hits.

[20] The comparison of satellite estimates against raingauges suffers from the fact that the former correspond to a grid box of 0.25° × 0.25° and the latter to point measurements. This difference affects the comparison much more during summer where the major part of rain in the Central and Eastern Mediterranean is produced by relatively localized thunderstorm activity, than during winter, when mid-latitude low pressure systems tend to produce much more widespread precipitation. As a result, the constellation of satellites depicts convective activity within grid boxes, but the chance to measure this localized convection by raingauges is limited. Moreover, one should note that for some precipitation thresholds (namely for low thresholds) the algorithms applied to satellite data may also suffer from retrieval errors as discussed in Bauer et al. [2002].

[21] Figure 3 presents some quantitative evaluation of the precipitation estimates. The quantity bias (Figure 3a) for the low and medium precipitation is small overall, ranging from 1.5 mm (in the 0.1–2.5 mm range) to −1.4 mm (in the 5–10 mm range). For the ranges of rain greater than 10 mm and for all seasons of the year the QB has large negative values, indicating a pronounced underestimation of the high precipitation amounts by the satellites. Again, the fact that satellite estimates refer to an average over a grid box and raingauge observations to point measurements, affects the results, especially for the quantitative verification of rain accumulated at 12-h intervals. MAE scores show the same increasing trend (Figure 3b), especially for the highest precipitation ranges. Specifically, the 12-months MAE increases from 2.6 mm at the 0.1–2.5 mm range to 20.4 mm at the 20–40 mm range, while the respective values for the dry period are 3.8 mm and 20.6 mm and for the wet period 1.9 mm and 20.3 mm. It should be noted however, that the percentage of error for the medium to high precipitation ranges is smaller than the percentage errors at low ranges of the observed rainfall. Namely the relative percent error for the whole 12-month period is 75% for the precipitation range 10–20 mm and 65% in the range 20–40 mm. Finally, it should be noted that the statistical scores for the highest precipitation ranges are based on a relatively small number of pairs compared to the number of pairs for medium and low thresholds and this feature should be taken into consideration.

Figure 3.

(a) Quantity Bias (QB), (b) Relative percent error (%) for the 12-month period (scaled following the left y-axis) and Mean Absolute Error (MAE) in mm (scaled following the right y-axis), for 6 precipitation ranges.

4. Final Remarks and Prospects

[22] A statistical verification of high resolution precipitation estimates from satellite data has been conducted, for accumulated rainfall in 12-h intervals, for a one year period over the Central and Eastern Mediterranean.

[23] The qualitative verification of precipitation for the wet period of the year showed a slight underestimation of the frequency of rain events. For the same period of the year the false alarm ratio and the probability of detection scores are worse with increasing precipitation amounts. During the dry period of the year, the bias index always exceeds 1, indicating that the satellites detect rain events more frequently than the raingauges. This feature can be partly attributed to the different sampling method that affects more the results in cases of localized convective activity. It should be noted that the false alarm ratio is quite high for all precipitation thresholds, reaching a value of 0.85 (for the >20 mm threshold) during the dry period of the year.

[24] The calculation of quantity bias showed a net underprediction of precipitation accumulations at higher thresholds. The mean absolute error also shows an increasing trend with increasing observed precipitation, but at medium to high precipitation ranges, the percentage error is smaller than the percentage errors at low ranges.

[25] A promising outcome of this verification process is that the satellite estimates are able to give relatively reliable estimations of the frequency of rain events, especially during the wet period of the year. The information about the areas affected by rain as estimated by satellites can be used for the adjustment of initial fields used by regional weather prediction models. Namely, the information derived from the satellite estimates contains useful mesoscale details that can be inserted into the humidity fields. Such an approach has been recently used in numerical weather prediction studies, but the humidity adjustment was based on information from weather radar echoes [Gallus and Segal, 2001; Ducrocq et al., 2002]. This effort is obviously limited due to the radar coverage and therefore it would be tempting to develop a methodology for humidity adjustment of the initial conditions used in regional weather forecasting based on the information provided by the satellite estimates. This work is currently under way and the results will be presented in future publications.


[26] This work was supported by the Greek General Secretariat for Research and Technology (Excellence in Research Centers – MIS64563). The reviewers are acknowledged for their valuable comments.