Comparison of two approaches for downscaling synoptic atmospheric patterns to multisite precipitation occurrence



[1] The physical linkages between climate on the large scale and weather on the local scale allow the formulation of downscaling approaches for assessing the impact of climate variability at point locations. This paper presents a comparison between two such approaches applied for downscaling synoptic atmospheric patterns to point rainfall occurrences on a rain gauge network. The approaches evaluated are the parametric nonhomogenous hidden Markov model (NHMM) and the nonparametric k-nearest neighbor downscaling approach. The NHMM defines local-scale weather as a function of a discrete weather state that is Markovian and depends on predictor variables representing synoptic atmospheric patterns. As the model is defined parametrically, the number of parameters that need specification increases as one considers more discrete weather states. Consequently, parameter identification and generalization to ungauged sites becomes difficult. On the other hand, nonparametric resampling is attractive because of its efficiency and simplicity, being structured as a direct probabilistic relationship between the larger-scale climatic variables and the local-scale weather. Such a formulation offers a simpler alternative to the NHMM approach of using intermediate hidden weather state variables but is less capable of representing persistence introduced through Markovian assumptions in the NHMM. In the comparison presented here, we applied weather-state-based nonhomogeneous hidden Markov model and the k-nearest neighbor bootstrap to estimate precipitation occurrences at a network of 30 rain gauge locations around Sydney, Australia. Our results suggest that both the models perform well in representing spatial variations while they show a lack in representing temporal dependence at scales longer than a few days as exhibited through wet spell length characteristics. Local-scale features that are difficult to represent through the large-scale climate predictors are, as expected, not reproduced by either approach.

1. Introduction

[2] It is well known that there are strong physical linkages between climate on the large scale and weather on the local scale. Downscaling has emerged as a potential tool to relate atmospheric circulation patterns to surface variables for generation of series, for forecasting, for studying climate variability, and for predicting the regional climate in a changed environment. The background, developments, theory, applications, and limitations of downscaling are well documented in the literature [e.g., Wilby and Wigley, 1997; Xu, 1999; von Storch and Zwiers, 1999; Yarnal et al., 2001; Prudhomme et al., 2002]. It is no surprise that empirical downscaling approaches are becoming increasingly popular for gauging impacts of climate variability and changes at local and regional scales because of their relative simplicity and inexpensive computer requirements.

[3] Empirical downscaling procedures involve the use of a set of predictors, mostly atmospheric variables or variables that are often sourced as outputs of general circulation models, a set of predictands, mostly rainfall and temperature at point locations having long historical records, and a model that relates the predictors to the predictands at the timescale of interest.

[4] Recently, the idea of relating daily precipitation to synoptic atmospheric patterns has led to the development of weather state downscaling models. In weather state downscaling models, synoptic atmospheric patterns are the basis for classifying each day into the most likely weather state, precipitation being modeled using an appropriate probability distribution within each such weather state. Different versions of these models have been proposed by many researchers, including Bárdossy and Plate [1992], Hay et al. [1991], Hughes and Guttorp [1994], Wilby [1994], Wilks [1999], and Stehlík and Bárdossy [2002]. Once developed, these models can be used for ascertaining the rainfall pattern under a changed climate scenario, providing a basis for developing water management options to preclude the impact climate change may have.

[5] The nonhomogeneous hidden Markov model (NHMM) [Hughes and Guttorp, 1994] represents a general class of weather state downscaling models and relates a small set of large-scale atmospheric predictors to daily precipitation occurrence at multiple locations via a finite number of hidden or unobserved weather states. The NHMM identifies the most distinct spatial patterns in the multistation, daily precipitation occurrence record. In this way, it captures much of the spatial variability in daily precipitation and approximately the temporal variability through persistence in the weather states [Charles et al., 1999; Hughes et al., 1999]. Hughes et al. [1999] applied NHMM to relate atmospheric circulation to precipitation occurrences at 30 rain gauge stations in southwestern Australia and observed that spatial and temporal characteristics were modeled fairly well. NHMM has also been successfully applied to downscale rainfall amounts [Charles et al., 1999; Bellone et al., 2000], which were simulated sequential to the generation of the rainfall occurrences for the locations in the study region. Further applications of NHMM to downscale Commonwealth Scientific and Industrial Research Organisation general circulation model (GCM) and limited area model (LAM) atmospheric data in southwestern Australia have shown that daily rainfall probabilities, spatial patterns in rainfall occurrences, wet and dry spell length statistics, and the probability distributions of rainfall amounts at multiple sites are well reproduced [Bates et al., 1998, 2000; Charles et al., 2000].

[6] Nonparametric methods offer a different rational for downscaling climate variables to finer scales and have been used extensively for this problem in recent years [Rajagopalan and Lall, 1999; Brandsma and Buishand, 1998; Buishand and Brandsma, 2001; Yates et al., 2003; Beersma and Buishand, 2003]. These methods offer the alternative of developing the downscaling relationship without a priori assumptions on the joint probability distribution associated with the predictors and the downscaled predictands. These methods are parsimonious, provide a flexible framework, can easily be modified to include additional predictors, and have the ability to reproduce any observed functional relationship. The k-nearest neighbor bootstrap (KNN) is a technique that conditionally resamples the predictands from the observed record on the basis of the predictor variables used. The lack of any assumptions defining the joint distribution of the predictors and predictands helps ensure an accurate representation of features such as nonlinearity, asymmetry, or multimodality in the observed record of the variables being modeled. For multisite resampling, since the variables at these locations are simulated concurrently, dependence across space is accurately preserved.

[7] Young [1994] used nearest neighbor resampling to generate daily maximum and minimum temperatures and precipitation simultaneously in a physically consistent manner. An algorithm for bootstrapping time series considering Markovian dependence was developed by Lall and Sharma [1996]. Rajagopalan and Lall [1999] illustrated the improvements offered through the use of nearest neighbor resampling in comparison with a parametric time series model of Richardson [1981] for generating multiple daily weather variables for Utah, United States. Brandsma and Buishand [1998] applied the single-site nearest neighbor resampling to simulate daily precipitation and temperature conditioned upon the atmospheric variables for a number of stations in the Rhine basin. Buishand and Brandsma [2001] used nearest neighbor resampling for multisite generation of daily precipitation and temperature conditioned upon atmospheric variables at 25 stations in the German part of the Rhine basin. Harrold et al. [2003] applied the KNN method to generate daily rainfall occurrence at a single location and reproduced the long-term variability and low-frequency features of the rainfall record satisfactorily. The KNN technique was applied by Yates et al. [2003] to simulate daily and seasonal weather variables at multiple stations conditioned upon hypothetical climate scenarios. Beersma and Buishand [2003] generated multisite sequences of daily precipitation and temperature conditional on atmospheric predictors. Better results were found by including previous day rainfall and temperature as additional conditioning variables.

[8] The need for intercomparison of various downscaling techniques has been raised by many authors from time to time [Wilby and Wigley, 1997; Xu, 1999; Yarnal at el., 2001]. A majority of such studies compared dynamical downscaling techniques to statistical alternatives using current and GCM-simulated climate data sets. Kidson and Thompson [1998] compared the Regional Atmospheric Modelling System (RAMS) dynamical model and a regression-based statistical technique, at both monthly and daily timescales, and found little difference between the two techniques. Similar results have been reported by Murphy [1999] and Mearns et al. [1999]. The results of these studies suggest that at least for present-day climate the computational requirements do not favor the use of the dynamical model. However, when derived climate change projections were used as input to these models, the statistical and dynamical techniques produced significantly different predictions of climate change [Murphy, 2000; Mearns et al., 1999]. The authors concluded that the changes in the observed and simulated predictor/predictand relationships and omission from the regression equations of variables that represent climate change feedbacks, but are weak predictors of natural variability, could be responsible for the differences in the predicted outputs. Some of the differences could also be due to the climate change simulation exceeding the range of data used to develop the statistical model.

[9] There also have been some comparisons of different statistical downscaling techniques focusing on precipitation and temperature [e.g., Wilby and Wigley, 1997; Wilby et al., 1998; Benestad, 2001]. Results of these studies indicated that statistical downscaling approaches were successful in reproducing the wet-day occurrence and the amount distributions in the precipitation time series, but were less successful at capturing the long-term persistence in the time series.

[10] Extensions of comparison studies beyond primary climate variables (e.g., precipitation and temperature) again using dynamically and statistically downscaled precipitation and temperature output have also been reported [Wilby et al., 2000; Hay and Clark, 2003]. These studies concluded that if these series were not corrected for bias, statistical downscaling appeared to be the safer downscaling choice.

[11] Still there appears to be a paucity of studies focusing on the comparison of different statistical downscaling approaches. The NHMM and the KNN resampling approaches have been extensively used in the recent past [Charles et al., 1999, 2000; Hughes et al., 1999; Bellone et al., 2000; Bates et al., 1998, 2000; Rajagopalan and Lall, 1999; Brandsma and Buishand, 1998; Buishand and Brandsma, 2001; Yates et al., 2003; Beersma and Buishand, 2003] and are formulated on the basis of rather different concepts. Consequently, it is important that we address the specific limitations and advantages of each approach so as to be in a position to identify the situations where one or the other could be used in future applications. Of specific interest here is the ability of either approach to generate rainfall occurrences at point locations covering regions that are representative of the scale of a medium to large catchment. Thus the specific objectives of this study are (1) to quantify the relationships between the large-scale atmospheric patterns and local-scale precipitation occurrence pattern at a network of stations in the Australian state of New South Wales by application of a weather-state-based parametric nonhomogeneous hidden Markov model (NHMM) and nonparametric k-nearest neighbor (KNN) downscaling approach; (2) to evaluate the performance of these approaches in terms of reproduction of spatial and temporal statistics using graphical and mathematical interpretations; and (3) to perform a cross-validation analysis to check the usefulness of these approaches when applied to new data.

[12] The paper is organized as follows. Section 2 presents the methodological aspects of the two approaches. Section 3 presents details on the data used and the implementation of the two approaches. Section 4 presents the results of the application of these approaches in a network of stations in the study area. In section 5 we conclude with a discussion.

2. Methodology

[13] In the discussions that follow, we denote an n-site rainfall occurrence vector at time t as Rt, a vector of atmospheric predictor variables at time t as Xt, and the total number of observations as m. In general, the downscaling problem could be expressed as the conditional simulation of RtZt, where Zt represents a vector of variables at time t that in addition to atmospheric variables Xt, may also include other variables to impart persistence from one time step to the next. In the simplest case, Zt can contain Xt only, which would ensure simulation of Rt conditional to the atmospheric predictors alone. While this may simulate acceptable values of Rt, it may also lead to misrepresentation of day-to-day variations if the modeling is done at a point scale.

[14] Presented next are details on two different methods that aim to formulate an approximate representation of the conditional simulation of RtZt. The nonhomogeneous hidden Markov model (NHMM) approaches this formulation by assigning Zt = St, where St denotes a discrete weather state at time t that is modeled as an order 1 Markovian process, wherein transition probabilities of the Markovian process are modified by inclusion of atmospheric vector Xt at time t (StSt−1, Xt). The KNN method considers Zt consisting of Xt only in one formulation, while consisting of a variable representing the wetness state of the previous time step in addition to atmospheric variables Xt in another. Please note that we have denoted all multivariable vectors or matrices using bold and single variables or parameters using nonbold characters or symbols.

2.1. Nonhomogeneous Hidden Markov Model (NHMM)

[15] Hughes and Guttorp [1994] and Hughes et al. [1999] defined the general class of NHMM by the following assumptions:Assumption 1

equation image

Assumption 2

equation image

According to assumption 1, given the current weather state, rainfall occurrence is assumed independent of all past rainfall occurrences, weather state, and past atmospheric data. Assumption 2 asserts that the state of the weather at time t depends on the state of the weather at the previous time step and the current value of atmospheric variables. This assumption also defines the term “nonhomogeneous” used in the model definition, indicating that the state transition matrix varies in time as a function of the atmospheric variables.

[16] To parameterize P(RtSt, Hughes et al. [1999] adopted the autologistic model for multivariate binary data:

equation image

where ri is the rainfall occurrence for site i, i = 1, n, βsij is the “conditional log-odds ratio” of rain at station i to rain at station j (in state s) based on the probability distribution P(ri, rjri,−j, St = s), where ri,−j is the vector of rainfall occurrences at all stations other than i and j. This formulation for P(RtSt = s) defines the spatial model (HMM-spatial) since the spatial correlation of rainfall occurrences Rti at stations in each weather state is taken into account.

[17] Further simplification in equation (1) can be obtained by considering βsij = 0 for all i, j, and s. In this case, we have

equation image

where psi = exp (αsi)/{1 + exp (αsi)}. The psi gives the probability of rain at station i in weather state s. This formulation is referred to as the conditional independence model (NHMM-ind) for P(RtSt = s) since the rainfall occurrences Rti are assumed to be spatially independent, conditional on the weather state.

[18] To reduce the number of parameters of the spatial model, Hughes et al. [1999] suggested modeling βsij of equation (1) as a function of distance and directions between stations i and j as follows:

equation image

where dij and hij are the distances and direction between stations i and j, respectively. This formulation requires four additional parameters (b0, b1, ϕ, and e) for each state. In order to accelerate the computational procedure, they suggested estimating the two nonlinear parameters ϕ and e from a nonlinear least squares regression analysis of the empirical log-odds ratios before the start of the parameter estimation procedure.

[19] The following parameterization was adopted by Hughes et al. [1999] to estimate P(StSt−1, Xt):

equation image

where V is the variance-covariance matrix for the atmospheric data. The μij parameters represent the mean vectors of the atmospheric variables when the state of the weather at the previous time step was i and the current state of the weather is j, while the γij parameters can be interpreted as baseline transition probabilities. It is necessary to impose the constraints ∑jγij = 1 and ∑jμij = 0, in order to ensure identifiability of the parameters.

2.2. NHMM Parameter Estimation

[20] The parameters for the conditional independence model for P(RtSt) in equation (2) are estimated using a closed form solution. In all other cases a numerical optimization algorithm needs to be used. Hughes et al. [1999] used a two-step expectation-maximization (EM) algorithm [Baum et al., 1970; Dempster et al., 1977] to obtain maximum likelihood estimates for the conditional independence model using numerical optimization techniques and the forward-backward algorithm [Juang and Rabiner, 1991]. For the spatial dependence case, Hughes et al. [1999] used a modified two-step EM algorithm (referred to as EM-Monte Carlo maximum likelihood (MCML) algorithm in their paper) to estimate the parameters of the autologistic model (equation (1)).

[21] In this paper, we have used the recently proposed adaptive Metropolis (AM) sampling approach [Haario et al., 2001; Marshall et al., 2004] for estimation of model parameters for P(StSt−1, Xt). This approach requires fewer specifications on the part of the modeler and has distinct advantages over traditional Metropolis sampling when the variables being sampled are strongly interdependent. Model parameters for P(RtSt) are estimated using the same procedure as followed by Hughes et al. [1999].

2.3. The k-Nearest Neighbor (KNN) Downscaling

[22] Unlike the NHMM, where explicit relations prescribe the conditional dependence RtZt composed of RtSt and St∣(St−1, Xt), the k-nearest neighbor approach specifies the conditional probability P(RtZt) using nonparametric statistical techniques. Nonparametric techniques differ from parametric techniques, in that the form of dependence or the conditional probability distribution is not prescribed but ascertained on the basis of the historical record. The k-nearest neighbor approach estimates the conditional probability RtZt on the basis of the k-nearest neighbors of the conditioning vector Zt. In essence, the k patterns in the historical series of predictors that are most similar to the current pattern are found, and the k sets of corresponding predictands are specified as likely realizations the system may have. Lall and Sharma [1996] suggested that as a basic guideline, k can be chosen as the square root of the length of the data set. The Lall and Sharma [1996]k-nearest neighbor conditional probability density function (pdf) is expressed as

equation image

where p(i) is the probability that the ith nearest neighbor will be resampled and k is the number of nearest neighbors considered based on Euclidean distances from the conditioning vector Zt.

[23] To account for the seasonality in the hydrologic record, Rajagopalan and Lall [1999] and Sharma and Lall [1999] suggested using a moving window of length ℓ days, centered on the current day, and estimating the conditional probability density on the basis of the samples included within this moving window. The window length ℓ is one of the two specifications (the other being the choice of the number of nearest neighbors k) that are needed before the method can be used.

3. Application

3.1. Study Area and Data

[24] The study region is located around Sydney, eastern Australia, spanning between 147°E and 153°E longitude and 31°S and 36°S latitude (Figure 1). There are different types of synoptic-scale weather systems affecting the study region in different seasons of the year. In winter the most significant rainfall events involve air masses that have been brought over the region from the east coast low-pressure systems. Orographic uplift of these air masses when they strike coastal ranges or the Great Dividing Range often produces very heavy rain. Such an event was responsible for Sydney's heaviest recorded daily rainfall, in August 1986. Also, the cumulative effect of a series of cold fronts crossing the region from west to east over a period of a few weeks may produce heavy rains over the southern inland parts of the region. Troughs on or near the coast have been responsible for extremely high short-term rains such as those in Sydney in 1984 [Bureau of Meteorology, 1993].

Figure 1.

Map of the study area showing the locations of rain gauge stations and atmospheric data grid.

[25] For this study, a 43-year continuous record (from 1960 to 2002) of daily winter rainfall occurrence at 30 stations around Sydney, eastern Australia (see Figure 1), was used. The interstation distances between station pairs vary approximately from 20 to 340 km. Six winter months from March to August (184 days) were pooled together for analysis. Missing values at some stations (<0.5%) were estimated using the records of nearby stations. A day was considered as a wet day or dry day depending on whether the rainfall amount was greater or less than 0.3 mm, respectively [after Harrold et al., 2003; Buishand, 1978]. Commonly used tests of homogeneity were applied to the individual rainfall series, and the majority indicated that the data can be considered as homogeneous at the 95% level of confidence.

[26] Both downscaling approaches use predictor variables that signify the state of the atmosphere in predicting the rainfall pattern on the ground. In this study, atmospheric predictors were selected on the basis of consultation with climatologists, recommendations from similar downscaling applications in Australia and elsewhere [Wilby, 1994; Katz and Parlange, 1996; Busuioc et al., 1999; Zorita and von Storch, 1999; Bárdossy, 1997; Winkler et al., 1997; Saunders and Byrne, 1999; Weichert and Burger, 1998; Sailor and Li, 1999; Murphy, 1999; Charles et al., 1999, 2000; Hughes et al., 1999; Bates et al., 1998, 2000; Timbal and McAvaney, 2001; Bellone et al., 2000], and the results of an empirical model-comparison study that compared different predictor choices. On the basis of these recommendations and a study of the data, mean sea level pressure (MSLP), north-south (NS) and east-west (EW) gradients of MSLP, geopotential height (GPH) at 700 hPa, and NS and EW gradients of GPH at 700 hPa were selected as our candidate atmospheric predictors for downscaling. The final set of predictors was determined on the basis of an empirical model comparison exercise, presented later.

[27] The required atmospheric variables were extracted from the National Centers for Environmental Prediction (NCEP) reanalysis data [Kalnay et al., 1996] provided by the National Oceanic and Atmospheric Administration–Cooperative Institute for Research in Environmental Sciences Climate Diagnostics Center, Boulder, Colorado, United States, from their Web site at The NCEP reanalysis data set has been widely used by the climate community for generation and prediction of time series of weather variables in Australia [e.g., Bates et al., 1998, 2000; Charles et al., 1999, 2000]. These variables were available on a 2.5° latitude × 2.5° longitude grid over the study region, 4 times a day for the same period as the rainfall record, and were estimated over 5 × 5 grid nodes. As an observed rainfall value represents the total rainfall over a 24-hour period ending at 0900 local time (LT) in the morning, the available atmospheric measurements at 1700 LT on the preceding day were considered as representative of today's rainfall.

3.2. NHMM

[28] Prior to the application of either approach, one needs to have a strategy to select relevant predictor variables from the list of candidate atmospheric variables presented in section 3.1. In addition to the choice of predictor variables, the number of hidden states in the NHMM needs to be specified. For NHMM both these specifications are made using the Bayesian information criterion (BIC) [Hughes et al., 1999]:

equation image

where L is the log-likelihood, T is the total number of observations, and p is the number of model parameters. Since NHMM parameter estimation is computationally demanding, a more extensive cross-validation-based selection was not conducted. As BIC can be considered to be equivalent to cross-validation [Hastie at el., 2001], we feel confident of the model selection and predictor identification process that has been adopted here.

[29] While deciding upon the number of weather states to be included in the model, BIC is one of the many criteria used. Other considerations include the capability of the model to reproduce certain key features (such as spell length characteristics) in the observed data and the increase in the number of parameters induced by an increase in the number of weather states.

[30] The number of weather states was first determined by fitting NHMMs having three to seven states. Although the BIC suggested a large number of states (see Table 1), the seven-state model did not seem to improve the fit of the model to the important spatial and temporal distribution statistics of the data. As a result, the final model selected was a six-state NHMM with the following three atmospheric variables: GPH at 700 hPa, east-west gradient of GPH at 700 hPa, and north-south gradient of MSLP. Results of six-state NHMM-ind and NHMM-spatial models obtained using these three atmospheric variables are considered for final comparison.

Table 1. BIC-Based Comparison of Alternate NHMM-ind Configurations
Atmospheric VariablesNumber of Hidden StatesNumber of ParametersNegative Log-LikelihoodBIC
Hidden ParametersOutput Parameters
GPH, MSLP(NS), GPH(EW)3249088,998179,019
GPH, GPH(NS), GPH(EW)3249089,067179,157
MSLP, MSLP(NS), MSLP(EW)3249086,549174,121
MSLP, MSLP(NS), MSLP(EW)44812081,567164,642
GPH, MSLP(NS), GPH(EW)44812084,510170,528
GPH, GPH(EW)56015078,149158,183
GPH, GPH(NS)56015077,999157,883
MSLP, MSLP(EW)56015078,980159,845
GPH, MSLP(NS), GPH(EW)58015077,949157,963
MSLP, MSLP(NS), MSLP(EW)58015078,781159,627
GPH, GPH(NS), GPH(EW)58015078,932159,929
GPH, GPH(EW)69018075,669153,762
MSLP, MSLP(NS)69018077,897158,218
GPH, MSLP(NS)69018077,843158,110
MSLP, MSLP(EW), GPH(NS)612018075,752154,197
GPH, MSLP(NS), GPH(EW)612018075,076152,845
GPH, GPH(EW), GPH(NS)612018075,304153,301
GPH, MSLP(NS), GPH(EW)716821074,617152,627
GPH, GPH(EW), GPH(NS)716821074,698152,789

3.3. KNN Model

[31] After analyzing the sensitivity of our model to different choices of width of moving window ℓ, a value of ℓ = 15 days was chosen for use in our application. Similarly, an analysis was performed to find out the optimal value of k, the number of nearest neighbors, and consequently a value of k = 20 was adopted for use in the present study. The atmospheric variables were finalized using leave-one-out cross-validation (L1CV), the test statistic being the L1CV mean-square error. The L1CV indicated that GPH at 700 hPa, east-west gradient of GPH at 700 hPa, and north-south gradient of MSLP were the optimal atmospheric predictors for the KNN downscaling model. The same predictors were identified as optimal for the NHMM on the basis of the Bayesian information criterion. It should also be noted that these predictor choices were marginally superior to the alternate configurations assessed, suggesting that multiple predictor sets may result in similar performances. This was reflected in the results of the NHMM too, reported earlier in Table 1. The series of predictors were standardized by subtracting the mean and dividing by the standard deviation of all the values falling within the moving window.

[32] The study considers two formulations of the KNN. The first one considers rainfall occurrences directly conditioned upon atmospheric variables, hereinafter referred to as the KNN-atm model. To improve representation of temporal dependence of the rainfall occurrences at each station, in the second formulation, previous-day average rainfall occurrence of the region is included as an additional predictor. This formulation is hereinafter referred to as the KNN-tmp model.

3.4. Statistical Performance Measures

[33] While the models and predictor identification, as described before, were based on the full historical record, model results presented in section 4 were evaluated using cross-validation. Cross-validation was especially of interest in evaluating NHMM performance to assess the stability of model outputs given the large number of parameters that need to be specified. Cross-validation was performed by dividing the complete record of 43 years into blocks of 6 years (the last block being 7 years long) and estimating the rainfall for each block one at a time, with the model being formulated using the remaining data. For example, in the first instance, both models were formulated using data from years 7–43 and applied to predict the rainfall for years 1–6. This was then repeated for each of the remaining blocks, and performance measures were computed. In all the results that follow, the statistics reported have been ascertained by generating 100 realizations of the rainfall occurrences from both the NHMM and the KNN. The models are compared for their ability to simulate the observed spatial and temporal characteristics of rainfall, including those of importance in water resource management.

[34] The graphical comparison of different models was performed on the basis of (1) log-odds ratio, a measure of the spatial correlation in the rainfall occurrence; (2) seasonal wetness fraction, a measure of the frequency at which wet days are simulated within the season; (3) wet/dry spell duration, a measure of the temporal dependence that exists at any given site from one day to the next; (4) maximum length of wet/dry spell duration, a measure of the extreme weather states that are important in representing hydrologic extremes; and (5) solitary wet days or wet days surrounded by dry days on either side, a measure of local-scale possibly convective cloud patterns that may not be well represented using the larger-scale climatic variables we have considered in our downscaling models. Numerical comparisons were based on the estimation of the root-mean-square error (RMSE) for the above statistics.

4. Results

[35] Various aspects of the rainfall occurrence patterns produced by NHMM and KNN downscaling models are compared in this section. It was interesting to find that the performance of both models was fairly similar in both calibration and cross-validation. In the case of NHMM, given the large number of parameters that need specification, this result was contrary to our initial expectations and reflects on the stability of the model structure. Taking note of the similarity between cross-validation and calibration results for both models, our presentation relies mostly on the cross-validation performance of each approach.

4.1. Spatial Dependence

[36] Accurate reproduction of the spatial dependence of long spells of rainfall events is needed for correct simulation of river discharges over a large area. The log-odds ratio [Agresti, 1996; Bishop et al., 1975] reflecting the spatial correlation between rainfall occurrences at each pair of stations provides such a measure. This ratio is defined as

equation image

where LRi,j is the log-odds ratio between i and j pair of stations; p11i,j, p00i,j, p10i,j, and p01i,j are the joint probabilities of rain at both stations, no rain at either of the stations, rain at station i and no rain at station j, and no rain at station i and rain at station j, respectively. This measure of spatial dependence is more apt for representing dependence between discrete variables such as rainfall occurrence. A high value is indicative of better defined spatial dependence between the variables.

[37] Figure 2 presents observed and modeled log-odds ratios for the NHMM and the KNN model at all stations. Each point on the graph indicates the ratio evaluated for a pair of rain gauge stations. As can be seen from the figure, the log-odds ratios are modeled less accurately by NHMM-ind, especially when the observed correlation is high. The common weather state seems to explain much of the correlation, but additional unexplained local spatial correlation remains. NHMM-spatial, which considers the spatial dependence in the form of distance and directions between rain gauge stations, leads to a better spatial representation than the NHMM-ind. As the KNN model considers precipitation occurrences concurrently at all the stations, the dependence between the stations is largely preserved by both formulations of the model.

Figure 2.

Comparison of observed and model averaged log-odds ratios at each station estimated using cross-validation.

4.2. Seasonal Wetness Fraction

[38] Reproduction of the total number of wet days in a season or the frequency of a wet day and day-to-day occurrence of the rainfall is essential in simulating the seasonal water balance in any modeling exercise in which the downscaled rainfall may be used. Figure 3 presents the scatterplots of observed and modeled wetness fraction (total wet days divided by the total number of days in the season) at all stations for different models. While NHMM-ind and NHMM-spatial provide a good fit to the data, KNN-atm and KNN-tmp models slightly underestimate the wetness fraction at all stations. The KNN-tmp model, which is aimed at maintaining day-to-day persistence of the daily rainfall by considering the average rainfall state of the previous day as one of its predictor, apparently does so at the expense of introducing a bias in its representation of the overall seasonal wetness. The RMSE associated with the total number of wet days per season is presented in Table 2. Also presented is the RMSE of daily rainfall occurrences, which considers the variations in observed and estimated rainfall occurrences on each day as compared to the wetness fraction that represents the average number of wet days over a period. The KNN-atm and KNN-tmp models represent this statistic more accurately as compared to NHMM-ind and NHMM-spatial (Table 2).

Figure 3.

Comparison of observed and model averaged wetness fraction, at each station, estimated using cross-validation.

Table 2. Cross-Validation Root-Mean-Square Error (RMSE)
1number of wet spells of 2 days per season2.372.382.512.48
2maximum wet spell length (in days) per season2.182.222.642.22
3total number of wet spells per season4.984.986.814.12
4number of dry spells of two days per season2.032.022.382.03
5maximum dry spell length (in days) per season9.439.279.768.86
6total number of dry spells per season4.924.926.774.07
7number of solitary wet days per season5.195.388.324.38
8number of wet days per season10.6610.8410.9211.97
9daily rainfall occurrence0.5670.5670.5260.524

4.3. Wet and Dry Spells

[39] Sustained wet and dry periods are of prime concern in catchment management studies. Figure 4 and Table 2 present the scatterplots and RMSEs of observed and modeled average number of dry and wet spells (of length 2–6 days for wet spells and 2–9 days for dry spells) per season for each station. As can be seen from the plots, these statistics are modeled satisfactorily by all the models. Figure 5 presents the scatterplot of observed and model averaged number of wet spells of duration greater than 6 days and dry spells of duration greater than 9 days, per season, for each station. All the models reproduce fairly well the dry spells of duration greater than 9 days; however, they are less successful in reproducing the wet spells of longer durations. It should, however, be noted that these are intrinsically rare events (on average, occurring once in every two seasons) and therefore results need careful interpretation. Similar results have been reported by Wilks and Wilby [1999] and Wilby [1994]. As KNN-atm model considers the current day's rainfall occurrence as a function of the current day's atmospheric variables only, there is no guarantee that wet or dry spell characteristics will be reproduced adequately. The KNN-tmp model, which considers the regional average of estimated rainfall state of the previous day as an additional predictor variable, offers some improvement to the fit of the wet and dry spell duration characteristics (Figures 4 and 5 and Table 2). Other statistics of importance, related to wet and dry spells considered in the study, include total number of wet and dry spells in a season and number of wet and dry spells of 2 days in a season. RMSEs of these statistics are almost similar for NHMM-ind and NHMM-spatial. The KNN-tmp model is more successful in capturing these statistics as compared to KNN-atm model and either of the NHMMs (see Table 2). Other related statistics of interest include maximum length of continuous wet and dry spells in a season. Figure 6 presents the distribution plots of these statistics for all the models. Neither of the models is successful in capturing these statistics adequately. However, the KNN-tmp model better reproduces these statistics as compared to KNN-atm and either of the NHMMs. RMSEs of these statistics are similar for all the models (Table 2). The RMSE scores reveal that the number of wet days per season is better reproduced by NHMM as compared to either of the KNN models. However, the sequencing of these wet days that defines the wet spell is better preserved by the KNN-tmp model.

Figure 4.

Comparison of observed and model averaged number of wet and dry spells per season at each station, estimated using cross-validation.

Figure 5.

Comparison of observed and model averaged number of wet (greater than 6 days) and dry (greater than 9 days) spells per season, at each station, estimated using cross-validation.

Figure 6.

Comparison of observed and model averaged distribution statistics of maximum (a) wet and (b) dry spell lengths per season, at each station, estimated using cross-validation.

4.4. Solitary Wet Days

[40] Buishand [1978] and Chapman [1998] have shown that distribution of rainfall is different on solitary wet days (dry days on either side of a wet day) as compared to wet spells of longer length. This characteristic of the rainfall series may be crucial in representing enhancement of soil moisture in agriculture-related studies. Figure 7 and Table 2 present the scatterplots and RMSEs, respectively, for observed and modeled average solitary wet days per season for the models evaluated. As shown in the plots and also in Table 2, all the models are unsuccessful in reproducing this behavior satisfactorily. As this phenomenon is largely influenced by local climatological factors, a NHMM or KNN model that considers large-scale atmospheric variables as predictors is not able to preserve this characteristic. The KNN-tmp model, which considers the local rainfall conditions of the previous day in the form of a predictor variable, provided the least RMSE estimate (Table 2).

Figure 7.

Observed and model averaged solitary wet days per season, at each station for different stations for (a) NHMM-spatial (six states) and (b) KNN-tmp models estimated using cross-validation.

4.5. Grouping of Data in the Form of Weather States

[41] Although the weather states are the intermediate outcome of the NHMM, they can provide a useful means of analyzing the weather on a particular day under a weather state. This is the strength of the model that cannot be directly represented in the KNN, as the discrete state variables are not used. The Viterbi algorithm [Juang and Rabiner, 1991] can be used to identify the most probable sequence of states in the observed record so that each day is classified into one of the states defined by the NHMM. An average of the atmospheric variables on these days provides a means of assessing the physical significance of the weather state. This exercise was performed for the finalized six-state NHMM-spatial. Averaging the geopotential height at the 700-hPa field over all days classified into a particular state gave the predominant pattern associated with that state. The same procedure led to the predominant MSLP pattern associated with each of the six weather states. These weather states identified by NHMM correspond to particular precipitation patterns. One may compare these atmospheric patterns to the corresponding rainfall patterns. Figure 8 shows the contour plots for MSLP, GPH, and also the precipitation patterns associated with first, third, and fifth weather states. State 1 corresponds to the high precipitation probability at all stations and is the most common winter pattern associated with the low pressures in the east and west parts of the region. State 5 is characterized by a dominant high-pressure system over the study region, which corresponds to a low precipitation probability at all stations. This is the most common weather state, occurring 38% of the time. The other atmospheric patterns are consistent with the observed precipitation patterns. These plots suggest that some of the weather states might be regarded as substates of the dominant weather pattern. For example, we find that state 2 corresponds to a low probability of rainfall along the coastal stations and very low probability of rainfall at inland stations, whereas state 3 indicates high probability of rainfall along the coastal stations and very little probability of rainfall at inland stations. These states correspond to the high-pressure system centered in the south part of the study region. Similarly, states 4 and 6 are characterized by the high-pressure system centered in the northwest part of study region and correspond to a high probability of rainfall at inland stations and low probability of rainfall at coastal stations. It should be noted that the inland and coastal regions appear to be separated by the Great Divide Range (Blue Mountains) in dictating the possibility of rainfall at inland stations being caused by a different mechanism or otherwise.

Figure 8.

(a) Rainfall probability, (b) sea level pressure field, and (c) geopotential height field distribution pattern for NHMM-spatial six-state model, evaluated using the calibration sample. The figure includes only three weather states: top row, state 1; middle row, state 3; and bottom row, state 5.

5. Discussion

[42] The study compared the application of the NHMM and KNN approaches for downscaling precipitation occurrence patterns at 30 rain gauge stations given synoptic atmospheric information. Some important conclusions of the study are as follows.

[43] 1. The results of cross-validation indicate that the selected atmospheric variables are able to represent the rainfall patterns effectively for this region. While the structure of NHMM requires many parameters to define the processes, results of cross-validation show that in spite of this, the model is well balanced and not overparameterized. The KNN model is essentially a data-driven model and therefore is expected to perform poorly when working with limited data. However, the results of cross-validation indicate that the model preformed well even with a limited data set and was able to reproduce the observed precipitation patterns for the periods for which data were withheld.

[44] 2. Although the NHMM and KNN differ in many respects in the manner in which they are formulated and implemented, both approaches reproduced fairly well the characteristics of the data in terms of spatial correlation and average wetness fraction. However, they were less successful in representing wet spells of longer durations. This result has importance in many applications (agriculture, catchment management, and flood mitigation) where the spells, and not single-day occurrences, are of interest. These findings indicate that the structure of either of these models in the present form is not adequate enough to address these characteristics properly and is consistent with the findings of Wilks and Wilby [1999] and Wilby [1994]. However, this interpretation needs to be taken with caution as the long spells occur relatively infrequently (on average, once in two seasons).

[45] 3. We also evaluated the performance of the models in terms of reproduction of solitary wet days and found that neither model reproduced this characteristic adequately. As this characteristic is primarily driven by local-scale factors or convective storm activity, it is unlikely that any model that considers large-scale atmospheric variables as predictors would preserve this characteristic well.

[46] 4. The NHMM classifies the weather of each day into the most likely weather state and hence tries to describe the physical linkages between atmosphere and the surface environment. The hidden Markov model assumptions simplify the temporal and spatial structures to be parameterized, since the common weather state accounts for some of the temporal dependence and much of the spatial correlation between rain gauges. However, the large number of parameters of the NHMM and computational complexities involved in their estimation procedures are the biggest limitations of the model. With a large network of stations the number of parameters grows almost exponentially with the increase in the number of states. This would render direct application of NHMM extremely computationally intensive, if not infeasible. The weather state in a NHMM typically indicates the predominant atmospheric patterns associated with the rainfall distribution at stations. Popular methods used for finalizing the order of a NHMM always settle for higher number of weather states, as ideally there could be as many states as the number of observations. Therefore all studies concerning NHMM suggest limiting the number of states on the basis of significant weather patterns and rainfall distributions. This, in a strict sense, imposes rigidity in both the spatial and temporal representation of rainfall. Also, the temporal dependence of rainfall occurrence at a station is governed by the assumption that the weather states follow a lag-one Markovian structure. This assumption helps in reproducing the temporal dependence of the observed record only if the underlying data structure follows the same pattern. In other cases, in order to preserve the longer temporal dependence, the model structure needs to be modified suitably. This, however, in turn will attract more assumptions and more parameters and will introduce more complexity in terms of model structure and parameter estimation procedures. Owing to the already complex structure and large number of parameters of NHMM, the effort involved in further modification can be considerable.

[47] 5. The KNN approach as presented here considers the rainfall occurrence at stations directly conditioned upon the atmospheric variables and therefore cannot explain the linkages between the atmosphere and the surface environment in the form of a weather state as offered by NHMM. Also, as the approach resamples from the observed record, it requires long lengths of available observed record, especially in cases where many predictor variables are involved. Another limitation of this approach is that as it resamples from the historical observations, the exact observed record is always simulated and if we are to simulate the variable simultaneously at a number of stations, the patterns as observed in the past can be exactly reproduced. However, being simple and straightforward, the approach offers an attractive alternative to traditional parametric approaches. Issues such as usefulness of additional predictor variables and representation of longer-duration temporal dependence can be easily accommodated in KNN without introducing additional complexity.

[48] 6. It should be noted that while parametric methods offer the advantage of using defined parameter values that can possibly be regionalized and used in areas where the model may not have been fitted, this is not really a practical option in the case of the NHMM given the large number of parameters it uses. We feel that if regionalization is indeed required, the simplicity of the KNN formulation can easily accommodate modifications that enable estimation at unsampled locations, using neighbors sampled on the basis of the KNN logic in space in addition to time.

[49] 7. The number of weather states in the NHMM is kept small in order to reliably estimate the many parameters in the model. This has the effect of limiting the representation of rainfall that may not be characterized by the climate patterns associated with each state. Furthermore, cases where rainfall is a result of multiple mechanisms that are represented through a combination of states are again not represented. The KNN resamples the entire spatial vector of rainfall conditional to the atmospheric inputs supplied. Consequently, even if the rainfall has resulted from two or more causative mechanisms that correspond to multiple NHMM states, the full rainfall distribution will be resampled with a probability specified by the KNN conditional pdf. We feel that the improved representation of the wet spell length in the KNN results in Figure 6 were a result of the above limitation in the NHMM.

[50] 8. Another point worth noting is that approaches such as the NHMM are season-based; that is, they model a part of the year assuming it to be homogeneous. However, if the intention were to use the model for different seasons or for a complete year, then in order to estimate the model parameters for these seasons, calibration would need to be carried out separately for all seasons. On the other hand, nonparametric approaches such as KNN are not required to assume the existence of a rigid seasonal boundary, choosing instead to represent seasonal variations through a flexible moving window.

[51] 9. NHMM is structured to simulate rainfall occurrence at multiple locations. Rainfall amounts of wet days are simulated by first modeling the rainfall occurrences and then estimating rainfall amounts on wet days using a separate procedure [Charles et al., 1999; Bellone et al., 2000]. The KNN model, on the other hand, can be easily adapted to downscaling the rainfall amounts [Buishand and Brandsma, 2001; Beersma and Buishand, 2003]. While the KNN does suffer from the limitation that only observed values can be simulated (hence preempting the simulation of extremes greater than what were observed), alternatives such as those suggested by Laio et al. [2003] or Lall and Sharma [1996] can be used. Laio et al. [2003] fit a linear/nonlinear trend among k-nearest neighbors and predict the new value on the basis of the fitted trend. Lall and Sharma [1996] propose resampling residuals from the conditional mean using the KNN resampling procedure, thereby ensuring that the values generated are not the ones that were from the observed record.

[52] 10. Downscaling models like the NHMM and KNN can also be used to study the effect of climate variability assuming that the relationship between atmospheric variables and surface environment is stable over time and GCM outputs are reliable and adequately represent the large-scale features of the atmosphere. Different realizations of the atmospheric fields included in the model can be obtained by repeatedly simulating the GCM under current climate conditions. These models can be used to transfer the effect of the variability at the synoptic scale to a point scale. The output of GCM runs under altered climate conditions can serve as input to these models. Thus the effects of the altered climate scenario on the local-scale precipitation processes can be studied by generating precipitation occurrences from these models. The KNN approach, however, has the limitation of parametric tuning of the model to accommodate hypothesized climate changes outside the training climate. Investigations of the use of these models for climate change assessment will be presented at a later date.

[53] 11. Alternate choices of predictors for both downscaling models did not result in a significant loss of performance as measured with respect to the BIC (in the case of the NHMM) or the leave-one-out cross-validation (in the case of the KNN). Also, the likelihood score (Table 1, for NHMM) and mean-square error (evaluated using KNN on the basis of leave-one-out cross-validation) for different sets of predictors indicate little difference in the performance of both models. Hence multiple predictor variable sets, each resulting in similar performances, could exist for both models.

[54] 12. The predictor selection was performed using the complete historical record. We feel that the use of cross-validation for selecting predictors on the basis of different subsets of data is not likely to make a change to the results reported. This assertion is based on the relatively stable and stationary attributes exhibited by both the atmospheric and the response variables when evaluated over shorter segments of the record.

[55] Several possible improvements to these models are currently under investigation, including more realistic spatial dependence structure and reduced parameterization of NHMM, introduction of weather state analogy in KNN, and more rigorous predictor selection criteria. Some simplicity in NHMM can be gained by dividing the study area into a number of homogeneous subregions and assigning a common parameter set to all stations within the subregion. To include the weather state analogy in KNN, inclusion of predictor variables explaining the lagged spatial distribution of rainfall patterns can offer some improvements. Work in these directions is in progress.


[56] The work described in this paper was partially funded by the Australian Research Council. Authors are thankful to the anonymous Journal of Geophysical Research reviewers for offering several insightful comments that helped improve the quality of the paper.