Scale-recursive estimation for multisensor Quantitative Precipitation Forecast verification: A preliminary assessment



[1] Precipitation is a highly heterogeneous process with considerable natural variability at scales ranging from a few meters to several hundreds of kilometers. This process is monitored with a variety of sensors (e.g., rain gauges, radars, and satellites) which provide direct or indirect measurements of precipitation at different scales. At the same time, physically based models at the storm, regional, continental, and global scales are used to predict precipitation and rely on the observed data for model verification, model initialization, and data assimilation. Because of the tremendous scale-dependent variability of precipitation fields, merging or comparing observations at different scales, or comparing model outputs at one scale with observations at one or more different scales, is not straightforward. This study explores the use of a recently developed scale-recursive estimation (SRE) framework for the problem of Quantitative Precipitation Forecast (QPF) verification using observations from multiple sensors. SRE can explicitly account for the multiscale variability of precipitation and the scale-dependent uncertainty associated with the model output and the multisensor observations. Special emphasis is placed on the specification of the multiscale structure of precipitation under sparse or noisy data. The results demonstrate the potential of SRE as a powerful tool for assessment of QPFs and also for multisensor data fusion and network design studies.

1. Introduction

[2] Studies on verification of Quantitative Precipitation Forecasts (QPF) are commonplace in the literature and serve several purposes. Among these are: (1) comparison of the performance of different models, (2) assessment of model improvements when new parameterizations are introduced, (3) assessment of model performance when resolution of the model is changed, and (4) estimation of errors of re-analysis products when model outputs are merged with observations to produce gridded fields for model initialization and other uses. Typically, the observations available for QPF verification are at scales different than the scale (grid size) of the model and comparison is not straightforward since the variability of precipitation fields strongly depends on the scale at which the fields are considered. For example, for a typical summer convective storm, the standard deviation of instantaneous precipitation more than halves if one goes from 2 km to 10 km pixel scale [e.g., see Tustison et al., 2001, Figure 1]. How this variability changes with scale is a function of the inherent characteristics of the storm (storm type) and the temporal integration scale (e.g., 5-min versus 1-hour accumulations etc.).

[3] In a recent study, Tustison et al. [2001] demonstrated the importance of accounting for the multiscale variability of precipitation when observations at one scale are compared with model output at another scale. They showed that using typical methods of QPF verification to change the scale of observations to the scale of model output (e.g., point-to-area conversion) or vice versa (area-to-point conversion) imposes a “representativeness error” which is non-zero even in the case of a “perfect” model. (“Perfect” model outputs were created in the numerical experiment by simply averaging the underlying field at several scales, and “perfect” observations were created by randomly subsampling the underlying field at various sampling densities.) As it was shown by Tustison et al. [2001], the representativeness error had significant magnitude (up to 50% of the spatial average of the precipitation field) and considerable scale dependency within the typical mesoscale ranges of 5–50 km. Also, the magnitude of the error was found to depend on the variability of the underlying field: the smoother the field (indicated by larger values of spectral slope), the smaller the representativeness error. It is stressed that the representativeness error results from the treatment of scale effects in the comparison method and is non-zero even in the case of a “perfect” forecast.

[4] We believe that a rigorous methodology which can explicitly account for the multiscale variability of precipitation is needed such that observations and model outputs at different scales can be compared or optimally merged while explicitly accounting for their scale-dependent variability and uncertainty. Such a methodology, based on scale-recursive estimation (SRE), is explored in this paper for the purpose of QPF verification from multisensor observations, typically available at different scales and with different uncertainties. This problem is illustrated in Figure 1.

Figure 1.

Illustration of the proposed framework for QPF verification. Discrepancy of scales between observations and model output presents a problem in QPF verification, data assimilation, and derivation of optimal merged products. Scale-recursive estimation (SRE) can account for the multiscale variability of the process and incorporate sparse observations and their measurement uncertainty at multiple scales to produce an optimal estimate of the process at any desired scale together with the uncertainty of this estimate. In this way, scale issues arising from the discrepancy of scales can be dealt with in a rigorous, computationally efficient and statistically optimal framework.

[5] The SRE technique was introduced in the signal processing literature [Chou, 1991; Chou et al., 1994a, 1994b] as a technique which can produce the best estimate (in terms of minimum variance of the estimation error) of the field at any desired scale, and also the uncertainty of the estimate, given sparse observations of the process and their uncertainties at different scales. SRE has its philosophical roots in the optimal recursive estimation technique known as Kalman filtering [e.g., see Anderson and Moore, 1979; Bras and Rodriguez-Iturbe, 1993] but instead of applying the estimation and recursion in time, it applies it in scale. The reader is referred to the original publications of Chou [1991] and Chou et al. [1994a, 1994b] for mathematical details and for demonstration of the success and computational efficiency of this technique. Since its introduction, SRE has found application in soil moisture estimation [Kumar, 1999], precipitation data assimilation [Primus, 1996], estimation of solute travel time distributions [Daniel et al., 2000], imaging and remote sensing problems [Fieguth, 1995], assimilation of remote sensing data [Daniel and Willsky, 1997], and estimation of satellite altimetry [Fieguth et al., 1995].

[6] In this work, SRE is explored for the problem of QPF verification using precipitation observations from multiple sensors. Special emphasis is placed on the selection of the model describing the multiscale variability of precipitation and on its parameter estimation from sparse or noisy data. Since the accuracy of SRE greatly depends on the above two factors, a simulation study is presented which quantifies the sensitivity of the SRE estimates on misspecification of model structure and model parameters. The results point out to the great potential of SRE for multisensor QPF verification, but also to the importance of performing controlled background sensitivity studies before this technique can be used with confidence in an operational setting.

[7] This paper is structured as follows. In the next section, a brief background on the SRE framework is presented while leaving some of the details for an appendix. Since SRE requires the specification of a model describing the multiscale structure of precipitation, section 3 provides information on a popular class of precipitation models (multiplicative cascades) and how these can be adapted for incorporation in the SRE framework. Section 4 presents the results of fitting two types of lognormal cascades to precipitation fields available from radars at 2 km resolution. In section 5, the sensitivity of SRE to multiscale model selection and uncertainty in the fitted parameters is investigated. In section 6, the effect of available observations (sparsity, measurement error, scales available) on QPF verification via SRE is studied and trade-offs between dense observations at a fine scale versus sparse observations at that scale and observations at another larger scale are quantified. Finally, conclusions and open problems are discussed in section 7.

2. Scale-Recursive Estimation

[8] A multiscale process can be represented on an inverted tree as seen in Figure 2. This tree can essentially be seen as a way of connecting information about the process at different scales. Each node on the tree corresponds to a unique combination of scale and spatial location and is given a location index λ. The index γλ is used to specify the value falling directly above that node on the next coarser spatial scale of the tree (called the parent node). Additionally, each node has a spatial scale index m(λ) which is the same for all nodes of the same spatial scale. The inverted tree starts at a root scale, which contains one pixel or node (called the root node) encompassing the entire spatial domain on which the estimation will be performed and which has scale index m(λ) = 0. In going to the next finer scale (i.e., increasing m(λ) by 1), the root node is divided into qx(λ) nodes in the x-direction and qy(λ) nodes in the y-direction, so that the root node contains q(λ) = qx(λ)qy(λ) children. These nodes are called children because each of them has the same parent node. Furthermore, each of these nodes may be further subdivided into their own children nodes across as many scales as is desired to characterize the multiscale process. For example, in a 2-d process where each scale contains two children in each direction (called a quadtree), qx(λ) = qy(λ) = 2 for all λ and each node contains q(λ) = 4 children. In general, the number of children at each scale may differ, allowing for a multiscale model that specifically accounts for spatial measurements at sets of scales, which do not fall on a quadtree. For a process evolving over a finite number of scales, the finest scale is commonly referred to as the leaf scale. Obviously, the nodes at the leaf scale have no children, as they represent the finest scale of the process.

Figure 2.

Quadtree representation of a multiscale process. A hypothetical example is illustrated above where sparse measurements of the process are available (denoted by solid circles) at two scales m(λ) = 2 and m(λ) = 1. Scale-recursive estimation (SRE) operates on the quadtree and involves an upward sweep (filtering) which consists of initialization, measurement update, variance propagation and merging and a downward sweep (smoothing) which allows the exchange of information between nodes of close proximity.

[9] The representation of a multiscale process on the inverted tree is achieved via a governing state-space equation, which specifies how the state at one scale relates to the state at other scales. This state-space equation may be manipulated so that the state evolves from fine to coarse (up the tree) or coarse to fine scales (down the tree). The model from coarse to fine scales appears more intuitive and is introduced first. Then, the model from fine to coarse scales is introduced as an inversion of the coarse to fine scale model. Additionally, a measurement model is introduced that relates measurements of the process and their uncertainty to the state at each node on the tree.

[10] The state-scale recursive equation specifying how the multiscale stochastic process evolves from coarse (γλ) to fine (λ) scales is of the form

equation image

where X(λ) is the zero-mean state of the system, A(λ) and B(λ) control the scale-to-scale composition of the process, and W(λ) ∼ N(0,1) is a driving noise which is independent of the state. The state can be a multidimensional vector composed of several variables. However, in this work, the interest is in estimating a scalar quantity (rainfall), and the state will be treated as a scalar throughout. It is noted that the state-space equation above can encompass a larger than first-order Markovian class of systems using the technique of “state augmentation” which has been extensively explored in state-space recursive estimation (e.g., see Szollosi-Nagy [1976] for several examples in the water resources literature). It is also worthwhile to note that although this formulation generally allows for the quantities A(λ), B(λ), and W(λ) to change with location at a given scale (i.e., to change in space), they are typically considered to depend only on the scale (i.e., they are constant for all spatial locations at a given scale) unless otherwise appropriate.

[11] Along with the estimate of the state, we are also interested in the scale-to-scale propagation of its variance in order to estimate the uncertainty of the estimates. Defining the variance of the state as Px(λ) = E[X2(λ)], using (1) and the fact that the state and noise term are independent, PX(λ) can simply be shown to evolve from coarse to fine scales as

equation image

This recursive equation shows that the variance of the state at one location is related to the variance of the state of its parent. Of course, before the state has been estimated, this variance is unknown and must therefore itself be estimated. Due to the recursive nature of (2), estimating the variance of the state at all scales may be accomplished by estimating the variance of the state at one scale, typically the root scale, and subsequently propagating it to all other scales through (2).

[12] The coarse to fine scale recursive model given by (1) can be inverted to give a model evolving recursively from fine to coarse scales

equation image

where W*(λ) ∼ N[0,Q(λ)] and F(λ) can be shown to take the form

equation image

Additionally, by taking E[X2(γλ)], the variance of the state can simply be shown to evolve from fine to coarse scales as

equation image

recalling that Q(λ) is the variance of W*(λ). Notice that if PX(λ) decreases with coarsening scale, and A(λ) is one or less in absolute value, then the magnitude of F(λ) will also be less than 1. This means that when going from fine to coarse scales (decreasing m(λ)), F(λ) will act to bring the state X(λ) closer to the process mean and reduce its variance PX(λ), as would be expected. It is noted that the above equations were presented for scalar states for reasons of simplicity. If X(λ) is a vector, the equations become more complicated and their form is given by Chou et al. [1994a].

[13] In order to incorporate the measurements of a process at different scales into this framework, it is necessary to form a measurement model that relates the measurements and the state of the system at a given location. The measurement model takes the form

equation image

where Y(λ) represents the measured quantity, C(λ) relates the state to the measurement, and V(λ) is the measurement error which is assumed normally distributed with zero mean and covariance R(λ), i.e., V(λ) ∼ N(0,R(λ)). Notice that this model incorporates a measurement uncertainty which may change with sensor and scale, as, for example, would be the case for rain gauges, radars, and satellites. When the measurement model (6) is combined with the fine to coarse evolution equation (3), a multiscale estimation, which respects the structure of the measurement error at every scale is obtained. Notice that C(λ) = 1 when the state coincides with the observable quantities; however C(λ) can be a complex often nonlinear transformation (a so-called “forward model” or “observational operator”) which relates the states of the system to the observable quantities.

[14] In order to compute estimates of the process and their uncertainty at every scale, the fine to coarse, coarse to fine, and measurement equations must be integrated into a single estimation framework. The multiscale estimates are computed from an upward sweep in which information is passed from one scale to the next coarsest scale, and a downward sweep which proceeds from coarse to fine scales. The upward sweep consists of initialization, measurement update, variance propagation, and merging steps. The downward sweep is much simpler conceptually and consists of only a smoothing step. It allows for the exchange of information between nodes with close spatial proximity, as spatially close nodes are not very far removed on the inverted tree and have contributed to the same upward sweep estimates of the state and its error variance. The upward sweep begins at the finest scale (leaf scale) and terminates at the coarsest scale (root scale). It utilizes multiscale Kalman filtering which incorporates observations and their uncertainties while estimating the state with (3) and variance with (5) at the next coarsest scale. In the downward sweep, information (state and error variances) is passed from coarse to fine scale through equations (1) and (2), respectively. The downward sweep begins at the root scale and terminates at the leaf scale. Used in combination, the upward and downward sweeps represent a generalization of the Rauch-Tung-Streibel smoothing algorithm [Chou et al., 1994a]. The details and equations of the upward and downward sweeps are given in Appendix A. They are also given by Chou et al. [1994a] and Kumar [1999].

3. Multiscale Models of Precipitation

[15] The scale-recursive estimation framework is based on specifying a model, which describes the multiscale variability structure of the process at hand. It allows for the incorporation of any model whose scale-to-scale evolution can be put into the recursive additive form given by (1). Several models have been proposed in the literature for describing the spatial variability of precipitation over a range of scales [e.g., see Gupta and Waymire [1990, 1993]; Lovejoy and Schertzer [1991]; Davis et al. [1994]; Kumar and Foufoula-Georgiou [1993a, 1993b]; Harris [1998]. A class of popular multiscale precipitation models, which naturally fit into the SRE framework is the multiplicative cascade models which are described below.

[16] A multiplicative cascade can be put into the recursive form

equation image

where xc(λ) is the value of the process at scale λ, xc(γλ) is the value of the process at the parent node, γλ, and wc(λ) are multiplicative cascade weights. If the weights come from a lognormal distribution the cascade is called a lognormal cascade. Here only canonical cascades are considered, i.e., cascades for which E[wc(λ)] = 1, that is, mass is conserved in an ensemble sense rather than exactly from scale to scale. Mass conservation can be enforced using microcanonical cascades. However, for these cascades no analytical relationships exist for the distribution of the multiplicative weights (except for a special case of beta-distributed weights [see Menabde and Sivapalan, 2000]), and thus it is not possible to implement them in the SRE framework.

[17] In order for a multiplicative cascade to be a good representation of a process, the process must be multiscaling. That is, the process must have a Fourier power spectrum that exhibits a power-law form

equation image

over a wide ranges of scales and its qth-order moments must take the form

equation image

where l is the spatial scale for which the moment is computed and L is the size of the field (e.g., l = 2 km and L = 128 km corresponds to moments computed from the 2 × 2 km2 field over a 128 × 128 km2 domain). In multiscaling moment analysis, an empirical K(q) curve is computed which can be fit to a theoretical K(q) function for a specific cascade type (e.g., lognormal cascade). Since K(q) will depend on the unknown cascade parameters, a least squares fit of the theoretical and empirical K(q) curves will produce an estimate of the cascade parameter(s) (the reader is referred to Harris [1998] for specific details of this fitting procedure). Alternatively, the cascade parameters for some cascade models can be determined from a plot of the variance of the log fields versus scale, if the specific form of the theoretical scale-dependence of the variance is known. For example, for a lognormal cascade on a quadtree, the change in variance between two adjacent scales is theoretically known to be constant (this will be seen below). Knowing this, allows a specific model to be fit from the empirical variance of the logs versus scale plot, which will yield the unknown parameter(s) of the lognormal cascade.

3.1. Lognormal Cascades

[18] For the lognormal cascade on a quadtree, the cascade weights at all scales come from the same lognormal distribution

equation image

where Z is a standard normal random variable (i.e., Z ∼ N(0,1)) and σc2 is the “log-variance” of the lognormally distributed cascade weights, i.e., the variance of the normally distributed log-transformed weights. The larger the value of σc, the larger the variance of the process. For example, a lognormal cascade with σc = 0, corresponds to a uniform field because the weight at every step in the cascade is one. In order for the coarse to fine scale recursive multiscale model given by (1) to be applicable for a lognormal cascade, the state X(λ) and the driving noise W(λ) must be zero-mean processes. Since neither the logs of the cascade values nor the logs of the cascade weights are zero-mean, this must somehow be accounted for, when this framework is applied to spatial rainfall. As can be seen from (10) the logs of the lognormal cascade weights are normally distributed and are given by

equation image

Recalling that Z ∼ N(0,1) and acknowledging the independence of σc and Z gives the mean of these weights as E[ln wc(λ)] = −σc2/2 and the zero-mean log weights may be written as

equation image

Applying this definition from the root scale (λ0) to increasingly finer scales, it is easy to verify that the zero-mean state of equation (1) may be written as

equation image

where, m(λ) is the index of the scale, as shown in Figure 2. For simplicity the equations are developed here for dyadic cascades where each m(λ) corresponds to a scale doubling, but can be extended to non-dyadic cascading. The lognormal cascade model can be put into the form of the SRE framework by using the zero-mean state given by (13) and selecting the parameters A(λ) and B(λ) in (1) to be

equation image

so that one obtains from (2) that the variance of the log-process is

equation image

3.2. Bounded Lognormal Cascades

[19] Recent studies have shown that rainfall is often better described by a cascade whose weights change with scale [e.g., Harris et al., 1998; Menabde, 1998]. For the bounded lognormal cascade on a quadtree, the cascade weights have the same form as those of the lognormal cascade except that now σc is a function of scale and thus is given by σbc(λ). Using the subscript “bc” to refer to bounded lognormal cascade, these weights may be written as

equation image

Furthermore, σbc(λ) is chosen so that the cascade weights follow a specified change with scale

equation image

where σ1 ≡ σbc[m(λ) = 1] and the parameter H controls how fast the variance of the weights decays with increasing scale index m(λ). Again, the state X(λ) and the driving noise W(λ) terms of (1) must be zero-mean in order for (1) to be applicable for the bounded lognormal cascade. Borrowing from what was done for the lognormal cascade, the zero-mean state must follow

equation image

Notice that this relation will reduce to that of (13) if σbc(λ) does not change with scale (i.e., H = 0), as one would expect since this scale-dependence of the weights is the only difference between the lognormal and bounded lognormal cascades. Additionally, following the same procedure as before, it can be shown that the bounded lognormal cascade model can be put into the form of the SRE framework by selecting the parameters A(λ) and B(λ) to be

equation image

so that one obtains from (2) together with the expression for σbc(λ) in (16) that the variance of the log-process can be expressed as

equation image

4. Model Fitting to Precipitation Fields

[20] The lognormal and bounded lognormal multiplicative cascade models were fitted to several hourly accumulated rainfall fields from summer convective storms observed by radar as shown in Table 1. Reflectivity maps were converted to rainrate images using a Z-R relationship of the form R = αZb, with α = 0.017, b = 0.714 [Smith et al., 1996] and where Z is in mm6/m3 and R is in mm/hr. Hourly accumulations were computed as simply the sum of the instantaneous rainrate images multiplied by the time between scans (5 minutes). The images of the accumulation fields for the four individual radar sites analyzed here are given by Tustison [2001] and Tustison et al. [2001].

Table 1. Estimated Parameters for Each of the Two Lognormal Cascade Models for Each of the Four Observed Hourly Rainfall Accumulation Fieldsa
Radar FieldbStormLN CascadeBounded LN Cascade
  • a

    The mean and standard deviation of the parameters over the four fields are also given to obtain a “representative” value of the parameters for field generation.

  • b

    Abbreviations are as follows: KAMA, Amarillo, Texas; KEAX, Kansas City, Missouri; KICT, Wichita, Kansas; KTLX, Twin Lakes, Oklahoma.

KAMA3 June 19990.70.990.29
KEAX4 July 19950.660.970.33
KICT17 August 19940.440.590.23
KTLX17 August 19940.390.520.22
Mean 0.550.770.27
Standard deviation

[21] Model fitting was performed by finding the optimal value(s) of the multiscale model parameter(s) that minimized the weighted sum of the squared errors between the empirical (computed from the natural logs of the observations) and theoretical model variance (computed from equations (14b) and (18b)) as a function of scale. The weights were assigned inversely proportional to the standard error of the standard deviation of the log-field at each scale. A problem in working with the logs of the precipitation fields is the handling of zeroes. Unfortunately, rainfall observations often include a large percentage of zeroes, and any technique based on logs will be quite sensitive to how these zeroes are treated. One suggested treatment is to set ln(0) to a small positive value. In this work, the following value was used for ln(0)

equation image

where min Y(λ) is the minimum observation at scale λ and c is a constant. Here the value of c = 1 was selected (corresponding to replacing zeroes with values of the order of 10−3) after a sensitivity analysis on a range of c values was performed. The fitted multiscale model parameters are listed in Table 1.

[22] As can be seen from this table, most of the estimated parameters are fairly consistent between the rainfall fields, with only σ1 for the lognormal bounded cascade model showing significant change from storm to storm. Recall that σ1 measures the large-scale (i.e., parent node) variance of the weights σbc(λ), while H reflects the scale-to-scale change in the variability of the weights. For the sake of conciseness, only the variance versus scale curve for the KEAX field (other fields show similar trends) is presented in Figure 3. As can be seen from this figure, the bounded lognormal cascade does a better job (as measured by residual errors) of representing the variance as a function of scale for this field (and the other three fields) than the simple lognormal cascade model. This makes sense, since the two-parameter bounded lognormal cascade model is more flexible and can fit no worse than the one-parameter lognormal cascade model. If the lognormal cascade model was a good representation of the measured rainfall, then the bounded lognormal cascade would reduce to the one-parameter lognormal by forcing the parameter H to be zero and the parameter σ1 would be equal to σc. Despite its shortcomings for rainfall modeling, the lognormal cascade continues to be used in the precipitation literature and was the model used in the SRE framework to assimilate radar and satellite microwave data in the study by Primus [1996]. It is noted that other estimation techniques such as that based on moment-scale analysis (i.e., determining the K(q) moment scaling exponent [e.g., Harris et al., 1996, 1997; Lovejoy and Schertzer, 1995, 1991]) can be used for fitting the cascades to observations. These techniques account for moments other than the second moment used above and can provide more robust estimates of the cascade parameters. However, they require complete fields of observations and thus are not applicable to sparse data sets for which the variance-based estimation remains the only choice.

Figure 3.

Plot of observed variance of the natural log of the field and associated lognormal and bounded lognormal cascade model fitted variances versus scale for the KEAX radar hourly accumulation image.

5. Sensitivity of SRE to Multiscale Model

[23] There is often significant uncertainty associated with the selection of the “best” multiscale model to represent a given precipitation field and also with the estimation of the model parameters once a model is selected. These uncertainties may be due to observational noise, observational sparseness in space and/or scale, or fundamental differences between the structure of the multiscale model and the rainfall process. Because at least one (and often all three) of these sources of uncertainty is likely to be present in selecting and fitting a multiscale model to observed precipitation data, it is vital to study the sensitivity of the scale-recursive estimation to these model misspecifications. In this section, the following three questions are addressed and quantified via numerical experiments:

  1. How sensitive is SRE to misspecification of model structure?
  2. How accurately can one estimate the true parameters of the underlying multiscale model from noisy observations?
  3. How sensitive is SRE to misspecification of the multiscale model parameters once the form of the model has been correctly identified?

[24] To address the first question, the KEAX hourly radar accumulations and the two fitted cascades of the previous section (LN with parameter σc = 0.66 and BLN with parameters σ1 = 0.97 and H = 0.33) were used. SRE was performed using both cascade models and observations at 2 km (KEAX field) and 16 km (spatially averaged KEAX field) to estimate the field at 8 km. The observations were considered very accurate with measurement error variance R(λ) = 0.002 at both scales (see equation (6)). (It is noted that this value of R(λ) corresponds to multiplicative measurement error ε ∼ N(0, s2) with s2 = 0.05 which is much below the observational error of radar estimates – see Appendix B.) The performance of the estimates was tested against the true field at 8 km (computed by averaging the 2 km field) via RMSE and bias. As can be seen from Table 2, the BLN cascade overperformed the LN cascade indicating, as expected, that providing a poor model for the multiscale structure results in poor performance of the scale-recursive estimation even if the observations are very accurate.

Table 2. Bounded Lognormal Cascade (BLN) Performancea
ModelRMSE at 8 km, mmBias at 8 km, mm
  • a

    BLN provides a better fit to the KEAX hourly data (see Figure 3) and outperforms the lognormal (LN) cascade in estimation of hourly precipitation at the 8 km scale. In both cases, the observations available for estimation were fields at 2 km and 16 km scales with very small measurement error. The same results apply to estimation at other scales as well.

LN (σc = 0.66)1.34−0.682
BLN (σ1 = 0.97, H = 0.33)0.320.086

[25] To address question (2), the uncertainty coming from proper model selection must be eliminated. This can only be achieved by working with fields generated with a known multiscale model. For that purpose, 100 realizations from LN and BLN cascades with parameters σc = 0.55 and σ1 = 0.77, H = 0.27 (the mean parameters of the 4 radar data in Table 1) were generated and the variance fitting method was used to estimate the cascade parameters. Each generated field was cascaded to 128 X 128 pixels to represent radar-observed rainfall at 2 km pixels over a 256X256 km2 domain. (The generation was done over nine cascade scales with an aggregation (dressing) performed over two upward scales to end up with fields considered to represent 2 km precipitation.) The ensemble statistics of the generated fields are summarized in Table 3. Figure 4 (left column) shows the histograms of the estimated parameters which quantify the parameter estimation uncertainty due to the natural variability of the process and the limited spatial extent of observations available for fitting. The most sensitive parameter to estimate seems to be the parameter σ1 of the BLN, which was found to have bias and considerable spread around the ensemble mean. When noise was added to the values of the generated cascades (to simulate the measurement error in the radar data; s = 0.4 was used which corresponds to the high end of observational noise as discussed in Appendix B), the estimation of the BLN cascade parameters was impacted significantly (see histograms in the right column of Figure 4). This implies that even if the true underlying structure of precipitation could be well described by a BLN cascade, the presence of measurement error (observational noise) might not allow an accurate estimation of the true parameters. This uncertainty in estimating the parameters of the multiscale model must be kept in mind in all applications of SRE, including QPF verification.

Figure 4.

Histograms of the estimated values of the cascade parameters from 100 realizations for lognormal (with σc = 0.55) and bounded lognormal (with σ1 = 0.77 and H = 0.27) cascades without measurement noise (left column) and with measurement noise (right column).

Table 3. Statistics of Generated Cascade Fields for 100 Realizations of the Lognormal Cascade (LN: σc = 0.55) and Bounded Lognormal Cascade (BLN: σ1 = 0.77, H = 0.27)a
StatisticCascade Ensemble Statistics (100 Realizations)
  • a

    All values are in millimeters.

Standard deviation2.511.89

[26] Having quantified the expected bias and uncertainty of the fitted cascade parameters, we turn our interest to question (3), i.e., quantifying how sensitive SRE is to this parameter uncertainty. To address this question, the following experiment was performed. An LN cascade was generated with parameter σc,true = 0.55. Then, SRE was performed using again the 2 km and 16 km fields as observations both considered very accurate (variance R(λ) = 0.05 in equation (6) as before) and SRE was used to estimate the fields at 8 km using an LN model with misspecified parameter σc (varied around the true parameter 0.55 in a range between 0.2 and 1.0). Figure 5 shows the results of this experiment and points out some interesting findings. First, underestimation of σcc < σc,true = 0.55) misspecifies the fields as smoother fields than they really are and thus the estimated 8 km fields have smaller variance than the true 8km fields. The opposite is true for σc > 0.55. The variance of the 8 km estimated field is equal to the variance of the true field only at σc = σc,true (see top two curves in Figure 5). Second, the best estimates (in terms of RMSE and uncertainty) are obtained when the specified parameter σc is close to the true value of 0.55. This quantifies the fundamental optimality property of SRE, which forms the underlying premise of the proposed method for QPF verification. It is noted that the RMSE curve is relatively flat at the uncertainty range of the estimate (here in the range of 0.45 to 0.65 as obtained from the top left histogram of Figure 4). This implies that if the model structure is specified correctly, parameter misspecification within the range of estimation uncertainty will not have a considerable impact on the accuracy of the scale-recursive estimates at any desired scale. From Figure 5, it is also observed that the bias of the estimated 8 km field is zero only when the specified σc is equal to σc,true; there is a small negative bias for underestimation of σc (i.e., when specifying smoother fields than they really are) and larger positive bias for overestimation of σc (i.e., when specifying rougher fields than they really are). Figure 5 provides all the information one would need in quantitatively assessing the performance of SRE due to misspecification of the multiscale model parameters. A similar analysis can be performed for a BLN cascade. However, interpretation of the results becomes more complicated since the curves in Figure 5 will become now surfaces in the two-dimensional space of the parameters σ1 and H.

Figure 5.

Effect of parameter misspecification on SRE estimates. The plot shows the RMSE, bias, mean uncertainty of the estimates at the 8 km scale as well as the standard deviation of the true and estimated fields as a function of the specified cascade parameter. It is seen that the best performance is achieved for σc = 0.58 which is very close to the true value of the underlying lognormal cascade σc,true = 0.55.

6. SRE Testing for QPF Verification

[27] Having established in the previous section the sensitivity of SRE to uncertainties about the multiscale model structure and the fitted model parameters, we now address questions related to the worth of observations and trade-offs between sampling densities and scales of available observations e.g., dense observations at a fine scale only versus fewer observations at that fine scale and simultaneous observations at a second larger scale. Although several combinations of sampling designs have been tested, we present here 5 cases only which we think elucidate the results. These cases are summarized in Table 4.

Table 4. Case Studies Illustrating the Worth of Observations on the Accuracy of SRE Estimatesa
CaseObservationsSRE Estimation at 8 km
BiasRMSEσest-fieldMean uncertainty of estimates
  • a

    It is seen that increase in sampling density and/or addition of a second scale of observations considerably improve the estimation accuracy in terms of bias, RMSE of the estimated field, variability of the estimated field σest-field (compare these values to σtrue = 3.01 mm) and mean uncertainty of estimates. All values are in millimeters. The mean of the KEAX hourly accumulation field at 8 km was 1.99 mm.

110% sampling at 2 km0.41.132.470.32
250% sampling at 2 km0.150.482.840.21
310% sampling at 2 km; 100% at 16 km0.170.842.800.3
410% sampling at 2 km; 100% at 32 km0.271.002.660.32
550% sampling at 2km; 100% at 16 km.0.100.452.880.21

[28] In all cases, the hourly-accumulated KEAX field was used with the fitted BLN cascade of parameters σ1 = 0.97 and H = 0.33. Sampling was done at random from the non-zero part of the storm only. Notice that 10% sampling corresponds to an average distance between observations on the order of ∼20 km. Estimation via SRE was performed at all scales, but only the 8 km results are presented here. Comparison of case 1 with case 2 (see Table 4) quantifies the improvement in the 8 km estimates when the density of observations at a specific scale increases. As was expected, increasing the availability of observations at 2 km (from 10% to 50% sampling) resulted in significant improvement in the estimation of the precipitation field at 8 km. Comparison of case 1 and case 3 quantifies the significant estimation improvement resulting from the addition of a large-scale observing sensor (e.g., satellite). Comparison of case 3 and case 4 demonstrates that if the second scale of observation becomes too large (e.g., from 16 km to 32 km), the additional information provided for estimation diminishes and estimation becomes less accurate. Comparison of case 2 and case 5 further quantifies the estimation improvement when a large-scale observing sensor is added. However, the improvement from case 2 to case 5 is much less than the improvement from case 1 to case 3 quantifying that if “enough” observations are available at a fine scale (e.g., 50%) the addition of the new larger-scale sensor does not help as much as when only a few observations at the fine scale are available (e.g., from rain gauges). It is noted that, in addition to the summary properties given in Table 4, the probability distributions and spatial covariances of estimation errors can also be plotted (e.g., see Figure 6 for cases 1 and 5).

Figure 6.

Images of the original KEAX hourly accumulation field at 8 km and estimated fields at the same scale for case 1 and case 5 of Table 4. The bottom plots show the error histograms for these two cases. It is noted that error fields here are computed by comparison of the SRE estimates at 8 km with the true 8 km fields (obtained by averaging of the 2 km fields). The comparison assesses the performance of SRE estimation. If, instead of these “true” fields, QPFs from a NWP model at 8 km were available, the error would quantify the performance of QPFs.

[29] All of the above findings make qualitative sense and point to the potential of SRE as a powerful tool for assessment of QPFs and also for multisensor network design studies. It is noted that other known methods of estimation, e.g., Kriging, do not allow easy treatment of observations at more than one scale. At the same time, SRE can easily incorporate any number of scales with no extra computational burden (e.g., see Chou [1994a, 1994b] for details on the computation advantages of SRE).

7. Summary and Conclusions

[30] Current methodologies for QPF verification are lacking in their ability to properly handle the multiscale nature of rainfall when observations at one or more scales have to be compared to the model output at yet another scale [e.g., see Tustison et al., 2001]. The scale-recursive estimation (SRE) method explored in this paper can incorporate rainfall measurements at multiple scales, as are often available (rain gauge, radar, satellite) together with their measurement error to produce optimal estimates of the field at any desired scale. By choosing the desired scale to be the model output scale, QPF verification can be performed by comparison of two fields at the same scale (see Figure 1). In addition, SRE provides the uncertainty of the estimates allowing for probabilistic assessment of the accuracy of QPFs. Since SRE is based on specification of a model describing the multiscale structure of precipitation, an analysis was performed here to assess the sensitivity of SRE to model misspecification. The results indicated that providing a wrong model structure (e.g., in our example LN cascade versus BLN cascade for hourly rainfall accumulations) can have significant effects on the estimation, and thus on the reliability of using the SRE method for QPF verification (see Table 2). However, if the model structure is specified correctly, misspecifying the parameters (within their uncertainty due to the fitting procedure, limited observations or noise in the observations, quantified in Figure 4) does not have a significant effect on estimation (see Figure 5) and thus SRE can be considered as a reliable and robust methodology for QPF verification. This study offers only a preliminary assessment of the potential of SRE for QPF verification. Despite the encouraging results, there are a lot of yet unsolved problems in adopting this methodology for real-time or operational applications. Some open problems for future research include: (1) exploration of SRE for robust model fitting given sparse observations at more than one scale (e.g., see the work of Luettgen and Willsky [1995]); (2) extension of the SRE framework to incorporate complex nonlinear relations (dictated by physics) between the observed quantity and the state of the system i.e., factor C(λ) in (6) to handle, for example, conversion of reflectivity to precipitation (for radars) or radiance to precipitation (for satellites); and (3) extension of the SRE framework to incorporate the temporal dynamics of the process.

[31] Furthermore, although this study focused solely on the problem of QPF verification (following our continuing interest in this problem [e.g., see Zepeda-Arce et al., 2000; Harris et al., 2001; Tustison et al., 2001]), it is obvious that the sensitivity analysis results presented here are pertinent to many other applications of SRE such as data assimilation, multi-sensor network design, derivation of initialization fields by merging observations and model output, and also estimation of the covariance of innovations (defined as the differences between observations and their estimates based on information carried by the model). This last issue is critical to any data assimilation effort as recently emphasized by USWRP [2000].

Appendix A:: Details of the SRE Framework

A1. Upward Sweep

[32] The upward sweep is the process by which information is carried up the inverted tree from fine to coarse scales. It consists of an initialization followed by three steps, which repeat at every scale. These steps are measurement update, scale propagation, and merging. At the leaf scale, m(λ) = mmax, the merged state (given a subscript “m” to refer to merged) at each leaf node is initialized to the global mean of the process, which is zero by definition (note that for non-zero mean processes one works with the mean-removed process), i.e.,

equation image

Since no observations have yet been incorporated into the estimation, the global mean is the “best guess” for the state. Also, the merged error variance of the state is initialized to the variance of the state at all leaf nodes, i.e.,

equation image

This is also a logical choice as an initial value for the error variance. Since no measurements have yet been used, the error variance of the estimates is equal to the variance of the process itself. The next step in the estimation method is the measurement update step, which provides an optimal procedure for updating the state and its error variance with the available measurements at a given scale. The updated state (given a subscript “u”) is given by

equation image

where K(λ) is the Kalman gain, a weight which is optimally chosen such that it minimizes the expected error variance of the state. The Kalman gain is given by

equation image

where R(λ) is the measurement error variance at location λ (see Chou [1991] for a proof). Notice that the multiplier of the Kalman gain in (A3) is actually the difference between the merged and observed values of the state. Thus, K(λ) is a weight which controls the relative contributions of the merged state (best estimate based on the previously available information) and the available measurement. The updated error variance of the state is also optimal in the minimum error variance sense and is given by

equation image

If no measurement is available at location λ, the values of the state and its error variance are not updated. That is, the merged state and error variance simply become the updated state and error variance. The measurement updated step and the associated Kalman gain may be easily understood when considering the lower and upper limits of the Kalman gain, K(λ) = 0 and K(λ) = 1, respectively. The case of K(λ) = 0 refers to a measuring device which is completely unreliable, since the error variance of the measuring device, R(λ), must be infinite when K(λ) = 0. This means that the available measurement provides no additional information about the state. In this case, we would expect the most current estimate of the state and its error variance to remain unchanged by the available measurement (exactly as if there was no measurement available). Examination of (A3) and (A4) reveals that this is certainly the case. The other limiting case of the Kalman gain occurs when K(λ) = 1. This corresponds to a “perfect” measurement, since R(λ) would be zero for this case. In this case, we would expect the state to become the measurement and its error variance to become zero, since the state would be known exactly. Again, examination of (A3) and (A4) reveals that this is the case. Now that the measurement-updated state and its variance have been determined, they may be used to find the values of the state and its error variance at the next coarsest scale.

[33] In the same way that a temporal Kalman filter uses past observations and their respective error variances to estimate the state of a system and its error variance for a future time step, the multiple scale Kalman filter uses observations and their respective error variances at finer scales to get estimates of the state and its error variance at increasingly coarser scales. Because of the discrepancy in the number of nodes at each scale (the number of nodes decreases as the scale index m(λ) decreases), there exists a problem with this propagation in scale. Each of the q(γλ) child nodes will predict the state Xp(γλ) and error variance Pp(γλ) (subscript “p” is used to represent propagated) of its parent, leaving us with q(γλ) predictions for each node at the parent scale. For now, let us ignore this problem and use each of the q(γλ) child nodes to estimate these values at the parent scale acknowledging that this problem will be addressed after the predictions are obtained from each child. The variance of the normally distributed noise W*(λ) is determined from the parameters of the coarse to fine scale evolving multiscale model and the variance of the state and may be written as

equation image

(see Chou [1991] for a derivation). At the next coarsest scale, the propagated state and its variance coming from the ith child (note that i is an index to differentiate between the different children of the same parent and ranges from 1 to q(γλ)) are given by

equation image


equation image

Notice that these equations correspond to the expected values of (3) and (5), respectively. Now, having q(γλ) predictions for each of the parent nodes, their combination to obtain a single predicted value for the parent will be considered.

[34] Our goal in the combination of these q(γλ) child node estimates of the state and its variance is to find estimates that minimize the expected error variance of the state. Such optimally merged estimates of state and error variance were introduced in this framework by Chou et al. [1994a]. Realizing that we have now moved to the parent scale, the merged error variance (given a subscript “m” for merged) takes the form

equation image

It can be seen that the merged error variance is a combination of the propagated error variances from each of the parent's children and the prior state variance coming from the evolution of (2). The merged estimate of the state is determined from the merged error variance and the propagated state and error variances from each of the parent's children. Specifically, the merged state is given by

equation image

As can be seen, the merged state is a weighted combination of the states propagated from each of the children where the weights are inversely proportional to the propagated error variances from the children. For more details, the reader is referred to the original publications of Chou [1991] and Chou et al. [1994a, 1994b].

A2. Downward Sweep

[35] These three steps of updating, scale propagation, and merging are carried out until the root scale is reached, upon which the upward sweep is terminated, and the downward sweep may begin. The downward sweep is a coarse to fine scale evolution beginning at the next finer scale from the root scale, and continuing to the finest (leaf) scale. Both the downward sweep estimates of the state and its variance are produced as the sum of its estimate from the upward sweep and a weighted difference between the upward and downward sweep estimates of its parent. These estimates of the state and its variance coming from the downward sweep are called smoothed estimates, because they come from averaging, a smoothing process. The downward sweep estimates, referred to as Xd(λ), are the final estimates of the state and its error variance from the multiscale model and are given by

equation image


equation image

respectively, where J(λ) is a weighting coefficient which relates to F(λ) and the measurement updated error variances Pu(λ) and Pu(γλ) through

equation image

(see Chou [1991] for a derivation). The downward sweep allows for the exchange of information between nodes with close spatial proximity, as spatially close nodes are not very far removed on the inverted tree and have contributed to the same upward sweep estimates of the state and its error variance. Although this is true for most spatially close nodes, there are some nodes that occupy close spatial locations but are quite far removed from each other on the multiscale tree (essentially the nodes nearby but on opposite sides of the spatial boundaries defined by the nodes at the largest scales). This is one of the drawbacks of the multiscale framework and has been addressed by Irving et al. [1997] with the use of overlapping multiscale trees.

Appendix B:: Observational Error for Radar Rainfall

[36] Rainfall intensity and radar reflectivity are related with the so-called Z-R relationship R = aZb [e.g., see Smith et al., 1996]. The error in this relationship is typically considered as multiplicative [e.g., see Smith and Krajewski, 1993], that is,

equation image

where ε ∼ LN(με,s2) is a lognormally distributed error process. Treating the rain rate transformed reflectivity Rt = aZb as the truth, a relationship between the true and observed rain rates can be formed as

equation image

Taking the logs of this equation transforms the multiplicative relation into an additive one given by

equation image

This equation is in the same form as the measurement equation of the scale-recursive model given by (6). The analysis of Smith and Krajewski [1993] suggests that the variance of the lognormal error process, denoted here by s2, is near 0.12 for a range of storms in various locations. Using the relationships between the moments of a lognormally distributed variable and its normally distributed log transform, one can easily verify that the corresponding variance w2 of the normally distributed noise term ln ε ∼ N(0,w2) can be written in terms of the variance of the lognormal variable as

equation image

Using this relationship gives the variance of the normally distributed variable as w2 = 0.10 corresponding to s2 = 0.12 found by Smith and Krajewski [1993]. In our analysis, measurement error with s2 = 0.16 (s = 0.4) was used to correspond to a high level of observational error (see section 5 and also Figure 4 when noise was added to observations). Accurate precipitation observations (i.e., observations with small variance R(λ) in equation (6)) were given a value of s2 = 0.05 which corresponds to w2 = R(λ) = 0.002 (see section 5).


[37] This research was partially funded by NSF (ATM-9714387 and ATM-013094), NOAA/NASA-GCIP (NAG8-1519), and NASA/TRMM (NAG5-7715). Computational resources were provided by the Minnesota Supercomputing Institute (MSI). We gratefully acknowledge the support of all these sponsors. We also thank Praveen Kumar of the University of Illinois, Urbana-Champaign, for invaluable discussions during the course of this work, and also Kelvin Droegemeier of the University of Oklahoma for providing us with radar data.