SEARCH

SEARCH BY CITATION

Keywords:

  • North Atlantic Oscillation;
  • change points;
  • time series;
  • trend estimation;
  • typhoons

Abstract

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

[1] Over recent years, considerable attention has been given to the problem of detecting trends and change points (discontinuities) in climatic series. This has led to the use of a plethora of detection techniques, ranging from the very simple (e.g., linear regression and t-tests) to the relatively complex (e.g., Markov chain Monte Carlo methods). However, many of these techniques are quite restricted in their range of application and care is needed to avoid misinterpretation of their results. In this paper we highlight the availability of modern regression methods that allow for both smooth trends and abrupt changes, and a discontinuity test that enables discrimination between the two. Our framework can accommodate constant mean levels, linear or smooth trends, and can test for genuine change points in an objective and data-driven way. We demonstrate its capabilities using the winter (December–March) North Atlantic Oscillation, an annual mean relative humidity series and a seasonal (June to October) typhoon count series as case studies. We show that the framework is less restrictive than many alternatives in allowing the data to speak for themselves and can give different and more credible results from those of conventional methods. The research findings from such analyses can be used to appropriately inform the design of subsequent studies of temporal changes in underlying physical mechanisms, and the development of policy responses that are appropriate for smoothly varying rather than abrupt climate change (and vice versa).

1. Introduction

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

[2] The problems of trend estimation and change point detection in climatic series have received substantial attention in recent years. Key issues include the magnitudes and directions of underlying trends (defined here as long-term changes in statistical properties; see section 1.1 ofChandler and Scott [2011]for a justification of this definition), and the existence (or otherwise) of abrupt shifts in the mean background state. The detection of abrupt discontinuities is an important step in characterizing climatic trends, because records may contain non-climatic artifacts due, for example, to undocumented changes in recording practice, instrumentation and station location: failure to account for such artifacts can lead to biased estimates of trends [e.g.,Yang et al., 2006; Menne et al., 2009; Fall et al., 2011]. Unfortunately it is also an extremely challenging statistical problem when the times of these potential discontinuities are unknown, since the theory underpinning almost all standard statistical test procedures breaks down in this case [Lund and Reeves, 2002]. Many techniques have been proposed for addressing the problem including t-tests [e.g.,Staudt et al., 2007; Marengo and Camargo, 2008], Mann-Whitney and Pettitt tests [e.g.,Mauget, 2003; Fealy and Sweeney, 2005; Li et al., 2005; Yu et al., 2006], linear and piecewise linear regression [e.g., Tomé and Miranda, 2004; Portnyagin et al., 2006; Su et al., 2006], cumulative sum analysis [e.g., Fealy and Sweeney, 2005; Levin, 2011], hierarchical Bayesian change point analysis [e.g., Tu et al., 2009]; Markov chain Monte Carlo methods [e.g., Elsner et al., 2004; Zhao and Chu, 2006], reversible jump Markov chain Monte Carlo [e.g., Zhao and Chu, 2010], and nonparametric regression [e.g., Bates et al., 2010].

[3] While there are many procedures in use, the purpose of this paper is to motivate wider use of flexible regression techniques. In particular, we highlight the availability of methods that allow for both smooth trends and abrupt changes, and that enable discrimination between the two. These techniques can give different (and more credible) results from those of classical parametric regression and change point methods. Of course, we do not claim a monopoly on the use of flexible statistical techniques; other relevant work with a similar goal includes Grégoire and Hamrouni [2002] and Gijbels and Goderniaux [2004]. We do, however, aim to motivate the use of techniques that are intuitive, flexible and for which user-friendly software is freely available so that implementation by non-statisticians is straightforward.

[4] We consider three case studies: the winter (December–March) North Atlantic Oscillation (NAO) index series for the period 1864–2010 (i.e., the Hurrell [1995] NAO index); an annual mean relative humidity series sourced from the NCEP/NAR Reanalysis 1 project for the period 1948–2010 [Kalnay et al., 1996]; and a seasonal (June to October) series of typhoon counts in the vicinity of Taiwan for the period 1970–2006 [Tu et al., 2009]. In the next section we briefly describe the methods used. Analyses of the winter NAO index, relative humidity and the typhoon count series are described in sections 3 to 5, respectively. Finally, a discussion and our findings are given in section 6.

2. Methods

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

2.1. Simple and Local Linear Regression

[5] Many climatic series consist of real-valued data and are regularly spaced in time. A commonly used approach to trend estimation is the linear model

  • display math

where Y1, …, Yn is the series of interest, the trend function μt = β0 + β1t is a linear function of the time index t and the errors εt form a sequence of independent, normally distributed random variables with zero mean and a common variance.

[6] In contrast with the linear trend model, nonparametric techniques impose minimal assumptions on the form of μt and hence allow the data to speak freely about any underlying changes in the mean. Local linear regression [Bowman and Azzalini, 1997] is one such technique, in which the trend function is assumed to be smooth so that μt can be approximated by a straight line in the neighborhood of any time point τ : μt ≈ μτ + βτ(t − τ) say. Estimates of the coefficients μτ,βτ are found by solving the weighted least squares problem

  • display math

where the weight function w(•; h) is the normal probability density function with mean 0 and standard deviation h so that observations distant from τ are downweighted. The estimate of μτ ( inline image, say) is then taken as an estimate of the underlying mean at time τ. This procedure is repeated over a range of values of τ to build the estimated trend function. A variability band indicating the size of two standard errors above and below the estimate can be constructed, and used to provide an informal assessment of uncertainty in the estimation. Reference bands to assess the suitability of particular models, such as constant mean or linear trend, can also be produced.

[7] The smoothing parameter (or bandwidth) h controls smoothness of the resulting trend estimate and is expressed in the same units as t. The sm package [Bowman and Azzalini, 1997] (also their R package ‘sm’: Nonparametric smoothing methods (version 2.2–4), http://www.stats.gla.ac.uk/∼adrian/sm, http://azzalini.stat.unipd.it/Book_sm) in the R programming environment [R Development Core Team, 2010] offers three automatic methods for the selection of h: cross validation (hcv), an approximate degrees of freedom criterion (hdf) and a corrected Akaike information criterion (haicc) [Hurvich et al., 1998]. Alternatively, local linear regression can be viewed as a linear filter and its squared gain function used to guide the final selection of h [Chandler and Scott, 2011, chapter 2]. If, for example, features on temporal scales of less than a few years are of little interest then h could be selected so that the squared gain is effectively zero at frequencies corresponding to these scales and close to 1 at frequencies corresponding to longer time scales.

[8] A key issue is whether a fitted regression curve represents a real long-term trend or whether it can be attributed to random variation. The sm package contains routines that enable a formal test of either no change or a linear trend in the null hypothesis, against the local linear regression model in the alternative hypothesis (for the theory underlying this test seeBowman and Azzalini [1997, section 5.2] or Chandler and Scott [2011, section 4.1.7]). Under a given null hypothesis, the observed significance level (p-value) for the test can be obtained from the quantiles of a scaled and shifted chi-square distribution. With respect to the selection ofh, it is important to reflect on the difference between inference (where the aim is to detect the presence of some effect of interest) and estimation (where interest is focused on the best possible estimate of the regression function). The literature is replete with automatic bandwidth selection procedures, all of which have strengths and weaknesses and, critically, it is usually extremely difficult to tell from the data alone whether the conditions are met for one method to be preferred over another. There are also the open questions of whether the bandwidths for inference and estimation should be taken as the same, and whether error criteria for the determination of an optimal bandwidth for estimation are appropriate for inference [Gijbels, 2003]. Therefore, rather than relying on a single bandwidth, we encourage the use of hcv, hdf and haicc and the squared gain function as a reference to establish a ‘reasonable’ range of values for h. A plot of the p-value as a function ofh is called the significance trace, and it assesses the sensitivity of the test results to the choice of h.

[9] Another issue is to determine whether there is evidence for any sharp discontinuities in the mean level of a climatic series. If an otherwise smooth trend function contains a discontinuity at time z, local linear regression can be used to estimate the trend separately on each side of the discontinuity, following ideas proposed by Hall and Titterington [1992] and Müller [1992]. To test for discontinuities, Bowman et al. [2006] propose the test statistic

  • display math

where zt ∈ {5.5, 6.5, …, n − 4.5} is the grid of time points for the evaluation of T, g = n − 9, inline image and inline image are local linear estimators from the data lying to the left and right of zt, and inline image is an estimate of the sampling variance of the difference inline image.

[10] Large values of the above test statistic suggest that one or more discontinuities are present, and p-values can be calculated under the assumption of independent normal errors with zero mean and constant variance. As before, a significance trace can be constructed to assess the sensitivity of test results to the choice ofh. Bowman et al. [2006] show via simulation that the procedure has good power to detect genuine discontinuities when they are present, and apply it to detect an abrupt change in the flow of the Nile River in 1899.

[11] This approach allows evaluation of the evidence for discontinuities in a data-driven way, without imposing specific functional forms (such as linearity, piecewise linearity, or step functions) on the underlying regression function (other techniques offering a similar approach include those ofGrégoire and Hamrouni [2002] and Gijbels and Goderniaux [2004]). Moreover, the approach acknowledges that a series may contain both discontinuities and a smooth trend. This is a substantial advantage over many popular techniques for discontinuity detection, which implicitly assume that any long-term change must be due to a discontinuity (this is true for several of the techniques reviewed in the Introduction to this paper). The approach is also open to the possibility that a series contains only discontinuities and no additional trend. In this case, the ‘smooth trend’ is a constant function.

[12] The reference distributions for the test statistics above are derived under the assumption that the error terms (εt) in (1) are independent and normally distributed with constant variance. In practice, normality is not a critical requirement in large samples; however, failure of the constant variance assumption can lead to incorrect p-values. This is potentially problematic for the analysis of count data (i.e., data that are only measurable with integers), such as the numbers of occurrences of rare events in specified time intervals. The Poisson process is often used as a starting point for modeling such series. The variance of a Poisson distribution is equal to its mean: thus, in the presence of trends in the mean of a count series, the variance is unlikely to be constant. However, the square root of a Poisson distributed random variable has a variance that is approximately independent of the mean [Davison, 2003, p. 59]. Thus, to apply the above regression framework to the typhoon count series in Section 4 below, we first take square roots of the counts (for an earlier example of this approach, see Dobson and Stewart [1974]). More recent techniques are available that can handle heteroscedasticity directly [e.g., Grégoire and Hamrouni, 2002]; in this article however, we retain the simpler approach of transforming the data because it can be implemented more easily using existing software. For all of our analyses, we use residual diagnostics to check for violations of the modeling assumptions [see, e.g., Draper and Smith, 1998].

[13] The final critical assumption underlying the test is that the (εt) in (1) are independent. With time series data, correlation may be present and if this is ignored then results regarding the automatic selection of h and the calculation of p-values and reference bands will be incorrect. Where necessary, a good strategy is to fit a simple time series model, such as a first-order autoregression, to the residuals from a local linear estimate with a carefully chosen smoothing parameter. The corresponding covariance matrix can then be incorporated into the distributional calculations. The technical details are beyond the scope of the present paper, where the examples do not require this amendment. For more discussion, seeChandler and Scott [2011, section 4.2.4] and references therein.

2.2. Pettitt Change Point Test

[14] For comparative purposes we use the nonparametric approach developed by Pettitt [1979] to detect change points in the winter NAO index, relative humidity and typhoon count series. The null hypothesis for this test is that the observations are independent and identically distributed. Suppose R1, …, Rt are the ranks of the t observations Y1, …, Yt in the complete sample of nobservations. For a two-sided test (i.e., one in which the alternative hypothesis does not specify the direction of change), the test statistic is defined by

  • display math

where Ut,n = 2Wt − t(n + 1) and inline image j = 1, …, t.When the observations are continuous-valued, thep-value forK = k, pk say, is approximated by pk ≈ 2 exp[−6k2/(n3 + n2)] if pk ≤ 0.5. While this approximation is useful for the analyses of the winter NAO index and relative humidity data, the typhoon count series considered herein contains only seven unique values and hence cannot be considered as continuous-valued. Thus we used bootstrap resampling [Efron and Tibshirani, 1993] to compute pk for the typhoon series.

2.3. Poisson Modeling

[15] Tu et al. [2009] applied two Poisson models to their typhoon count series: one in which the rate parameter was assumed to be time invariant, and the other in which a single change point was assumed with different rate parameters before and after the change point. Denoting by Xt the number of typhoons in year t, the first of these models has

  • display math

for x = 0, 1, 2, …; and the second has

  • display math

where λi > 0 (i = 0, 1, 2) are unknown rate parameters (the expected number of typhoons per year); and δis an unknown change point. We used maximum likelihood to estimate the rate parameters, and the likelihood ratio (deviance) test to compare the fit of the Poisson models. The usual chi-square reference distribution for likelihood ratio test statistics fails in change point problems, however [Davison, 2003, section 4.6]; thus we used bootstrap resampling to compute the p-value of the deviance test statistic [seeChandler and Scott, 2011, sections 3.4.3 and 3.6].

2.4. Statistical Power Analysis

[16] We performed a statistical power analysis (via simulation) to compare the abilities of the discontinuity and Pettitt tests to detect change points under various conditions. Power is defined here as the probability of detecting a discontinuity when it is present. For a realistic study, we base our simulations upon an analysis of the winter NAO index series. This series exhibits a complex nonlinear trend (section 3), which we estimate nonparametrically. The fitted trend function is then used to determine the residual variance estimate inline image. The simulated series are generated by adding independent Gaussian errors with zero mean to the fitted trend function, for three different levels of error variance inline image where c = {0.25, 1, 2.5}. We then insert jumps in the series. Several jump locations are considered, with three jump sizes at each location. One of the locations is in the middle of the series; the others at t = {15, 25, 49, 98, 125, 135}. The jump sizes used, namely Δ = {0, 1, 2, 3}, encompass zero and the root mean square successive difference defined by

  • display math

[17] Three values of h were chosen to represent small, moderate and large amounts of smoothing (section 3). For each of the above settings, 1000 experimental series are generated and the number of rejections of the null hypothesis at the 0.05 level recorded. An accurate test procedure is expected to reject the null hypothesis 5% of the time in the absence of a jump, that is when Δ = 0.

[18] The presence of trends in the experimental series could cause violations of the null hypothesis of the Pettitt test [see, e.g., Vincent et al., 2011]. Since our interest here is the detection of sharp discontinuities, one approach is to detrend the series prior to application of the Pettitt test [see, e.g., Rodionov, 2005]. In practice the true form of the underlying trend is unknown and trend removal methods such as differencing and the fitting of low order polynomials can change the error structure. For illustrative purposes, we assume that the underlying trend could be represented adequately by a linear regression model and assess the consequences of detrending on the power properties of the test.

[19] We also investigate the ability of the discontinuity and Pettitt tests to detect changes in a hypothetical series for which the conditions of the null hypothesis of the Pettitt test are met. Here, data are simulated from the trend function defined by

  • display math

where tc is the change point, Δ is the jump size at t = tc and m = − 2Δ/(n − 1). The settings used for error variance, Δ and h are the same as those described above. The results are presented in Tables 1 and 2 and discussed in section 3 below.

Table 1. Power Properties of the Discontinuity Testa
JumphExperimental SeriesHypothetical Series
c = 0.25c = 1.0c = 2.5c = 0.25c = 1.0c = 2.5
  • a

    Entries indicate the proportion of 1000 simulations for which the null hypothesis of no discontinuity was rejected at the 5% level.

05.00.0440.0480.0460.0480.0500.048
07.50.0640.0490.0520.0470.0600.057
010.00.1700.0740.0640.0640.0520.044
15.00.0700.0530.0480.0490.0600.060
17.50.1080.0600.0520.0700.0530.067
110.00.3170.0910.0730.1150.0580.059
25.00.1410.0650.0520.1340.0680.057
27.50.3360.0990.0620.2220.0860.049
210.00.6850.1640.1010.3970.0940.068
35.00.2980.0980.0650.2470.1070.059
37.50.7260.1730.0890.5700.1500.097
310.00.9450.3150.1360.8530.2350.096
Table 2. Power Properties of the Pettitt Testa
JumpExperimental SeriesDetrended Experimental SeriesHypothetical Series
c = 0.25c = 1.0c = 2.5c = 0.25c = 1.0c = 2.5c = 0.25c = 1.0c = 2.5
  • a

    Entries indicate the proportion of 1000 simulations for which the null hypothesis of no discontinuity was rejected at the 5% level.

00.7650.2050.0910.1770.0170.0000.0390.0470.041
11.0000.8910.4980.0380.0060.0000.3390.0930.058
21.0000.9990.9600.0010.0020.0010.8960.3360.147
31.0001.0001.0000.2410.0010.0011.0000.6370.285

3. Winter NAO Index

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

[20] The North Atlantic Oscillation (NAO) is the leading mode of winter climate variability in the North Atlantic region and a major source of climate variability. It is characterized by a meridional displacement of atmospheric mass and its state is usually expressed as a meridional sea level pressure gradient between the persistent anticyclonic region near the Azores and the persistent cyclonic region between Iceland and southern Greenland. A positive (negative) NAO state corresponds to stronger (weaker) westerly flow across the region and the storm track is displaced northward (southward). NAO-related impacts on winter climate extend from Florida to Greenland and from northwest Africa to northern Asia on seasonal to interdecadal time scales [see, e.g.,Hurrell, 1995; Hurrell et al., 2003; Fealy and Sweeney, 2005].

[21] We examine the annual series of the winter (December–March) index of the NAO based on the difference of normalized sea level pressures between Lisbon (Portugal) and Stykkisholmur/Reykjavik (Iceland) for the period 1864–2010, obtained from http://www.cgd.ucar.edu/cas/jhurrell/nao.stat.winter.html. This series changes from strongly negative values in the late 1960s to strongly positive values in the mid-1990s and then declines (Figure 1). Inspection of the series suggests that the variance of the series does not change noticeably with time despite the apparent temporal changes in mean level. Tomé and Miranda [2004] used piecewise linear regression to identify trends and change points in this series for the period 1880 to 2003. For minimum intervals between change points of 10 to 35 years, a change point in the early 1960s was found in all solutions. A second change point located between the late 1900s and early 1920s was evident for minimum intervals of 15 to 35 years. However, these findings may be an artifact of a methodology in which any trend that is not linear must be represented by a piecewise linear function. Similar criticisms apply to many other methods for change point detection. Fealy and Sweeney [2005] applied a Pettitt test to the winter NAO index series for the period 1900 to 2000. They asserted the existence of four change points, with the change point at 1980 having the highest probability.

image

Figure 1. Local linear regression curve with hhcv = 6.53 (thick line) and variability band (gray) for winter NAO index series. (Variability band provides an informal assessment of uncertainty in the estimated trend function.)

Download figure to PowerPoint

[22] The selected values of the smoothing parameter for local linear regression are haicc = 85.2, hcv = 6.53, and hdf = 12.6 based on the default value of 6 equivalent degrees of freedom (so that the fit is in some sense of equivalent complexity to a parametric model with six parameters) which allows a moderate amount of flexibility for the regression curve. The squared gain functions for local linear regression models with hcv and hdf are shown in Figure 2. The cutoff frequency for hdf is about 0.03 cycles y−1. Therefore we choose hcv since we are interested in estimating trend at near decadal as well as multidecadal scales, and restrict the horizontal axes of significance traces to the range 5 ≤ h < 10 as it provides an appropriate degree of smoothing: in this context, the choice haicc clearly oversmooths so as to eliminate almost all structure of interest. The local linear regression curve with hcv (thick line) and the associated variability band (gray) are shown in Figure 1. For this bandwidth the regression curve appears to be nonlinear and the variability band indicates that the uncertainty in the estimated trend function is reasonably small. Diagnostic checks (not shown) indicate that the model assumptions (independent, normally distributed errors with constant variance) are not unreasonable.

image

Figure 2. Squared gain versus log frequency curves for local linear regression models fitted to the winter NAO index series with hcv = 6.53 and hdf = 12.6. Vertical dashed line indicates a frequency of one cycle per 30 years.

Download figure to PowerPoint

[23] Figure 3 displays significance traces for the null hypotheses of no trend (i.e., a constant mean), linear trend and no discontinuity. It indicates: strong evidence against the null hypothesis of no trend (p-value < 0.007 throughout the considered range ofh); that the null hypothesis of a linear trend can also be safely rejected (p-values < 0.005); and that there is little evidence of any discontinuities (p-values > 0.16). Therefore we conclude that the mean level of the winter NAO series exhibits a smooth but non-monotonic trend, with little evidence for the existence of abrupt changes. Application of the Pettitt test to the same series yields ap-value of 0.15 for= 1168 at = 108 (i.e., 1971). This finding suggests that there is little evidence against the null hypothesis that the data are independent and identically distributed, and is consistent with the richer results obtained from local linear regression.

image

Figure 3. Significance traces for testing null hypotheses of no trend, linear trend and no discontinuity in the winter NAO index series.

Download figure to PowerPoint

[24] Table 1 reports the results of the power analysis for the discontinuity test, in the case when the jump location is in the center of the series. When the jump size is zero, the power is close to the nominal level of 0.05 for most parameter settings. While low for small jump sizes with large error variance, it increases rapidly as the settings become more favorable. For a given jump size it also increases with increasing amounts of smoothing.

[25] Table 2 reports the results of the power analysis for the Pettitt test, again for a jump located in the center of the series. For the experimental series and zero jump size, the power is unacceptably high suggesting numerous violations of the null hypothesis. In contrast the power levels for the detrended series are very low for moderate and large error variances, indicating that detrending can entail a substantial loss of power and hence failure to reject the null hypothesis when it is false. Comparison of Tables 1 and 2 indicates that the discontinuity test has much greater statistical power for the experimental series, principally because it allows a flexible trend whereas the detrending process can result in a loss of information. Finally, for the hypothetical series, when the jump size is zero the power is close to the nominal level of 0.05 for both tests and most parameter settings. However, the results also indicate that the Pettitt test is more powerful when the conditions of its null hypothesis are met. This is to be expected because the discontinuity test uses local rather than global information when evaluating evidence at each point of interest. If, however, the series is detrended prior to the application of the Pettitt test the power decreases markedly with increasing error variance. For example, with Δ = 2 the power drops to 0.684, 0.008 and 0.002 for c = {0.25, 1, 2.5}, respectively.

[26] The results from the simulations with jumps located away from the center of the series are not reported in detail: the power of the procedures was insensitive to the precise jump location unless the jump was close to either end of the series in which case the power was reduced. (This is to be expected since the behavior of the series in the shorter segment will always be estimated less precisely in such settings.) Moreover, the comparisons between the procedures were all qualitatively similar to those reported above.

4. Relative Humidity Series

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

[27] Water vapor is the most abundant greenhouse gas in the atmosphere and plays a key role in the hydrologic cycle. In the lower troposphere, it acts as the main supply for rainfall in all weather systems [Trenberth et al., 2005]. There are several metrics for defining the concentration of water vapor in the atmosphere, including relative humidity. Ambrosino et al. [2011]investigated the relationships between southern African rainfall and large-scale climate factors, one of which was monthly mean relative humidity series sourced from the NCEP/NAR Reanalysis 1 (NCEP-R1) project. They noted the presence of inhomogeneities in these series, particularly those coinciding with the beginning of the so-called satellite era in 1979. There have been two major changes in the observing system used for NCEP-R1: during 1948–1957 when the upper air network was being established; and in 1979 due to the introduction of satellite infrared and microwave retrievals [Kistler et al., 2001]. The discontinuities arising from these changes often occur in data-sparse regions in the southern hemisphere [see, e.g.,Grist and Nicholson, 2001; Pohlmann and Greatbatch, 2006].

[28] Here we apply the discontinuity test to annual mean 850 hPa relative humidity data for the NCEP-R1 grid point at 20°S and 17.5°E and the period 1948–2010. The data were obtained fromhttp://www.cdc.noaa.gov/data/gridded/data.ncep.reanalysis.pressure.html. This series exhibits large variability at decadal scales, and apparent sharp discontinuities as opposed to smooth changes in mean level (Figure 4). Using the bandwidth selection approach described above, we restrict our analysis to the range 2 ≤ h < 6 since it provides a reasonable degree of smoothing. Diagnostic checks (not shown) indicate that the model assumptions (independent, normally distributed errors with constant variance) are again not unreasonable. The significance trace for this range (not shown) indicates p-values <0.05 for 3.4 ≤h < 6 and <0.01 for 4.3 ≤ h < 6. This suggests that there is evidence against the null hypothesis of no discontinuity.

image

Figure 4. Local linear regression curve with h = 4 (thick line) and variability band (gray) for 850 hPa relative humidity series. Dashed vertical lines indicate the locations of the three potential change points 1960.5, 1979.5 and 1986.5. (Variability band provides an informal assessment of uncertainty in the estimated trend function.)

Download figure to PowerPoint

[29] We focus on the three time points where the evidence of discontinuities is the strongest and most sustained over the considered range of h. These time points can be identified as those with the largest contributions to the test statistic T defined by equation (3): they are at 1960.5, 1979.5 and 1986.5 (cf. Figure 4). Figure 5 shows the contributions of these three time points to |T| as a function of h. Application of the Pettitt test to the series yields a p-value <10−4 for = 700 at = 40 (i.e., 1987). This finding suggests that there is very strong evidence against the null hypothesis that the data are independent and identically distributed, and corroborates the timing of third change point detected using the discontinuity test.

image

Figure 5. Contribution to |T| for three candidate change points in the relative humidity series, as a function of h.

Download figure to PowerPoint

[30] Although a detailed examination of the differences between the groups of data bounded by the potential change points could be undertaken, for the purposes of our study we limit our attention to the identification of plausible causative factors. The potential change point at 1960.5 reflects the end of the era in which there was a general lack of quality controlled, electronically available upper air data prior to the International Geophysical Year in 1957–1958 [Stickler et al., 2010]. The change point at 1979.5 coincides with the launch of the first operational, polar-orbiting satellite in late 1978 (TIROS-N) and the NOAA-6 satellite in mid-1979. Similarly, the potential change point at 1986.5 marks the launch of the NOAA-10 satellite mission in September 1986 and the failure of the NOAA-9 microwave sounder unit in mid 1987 (http://science.nasa.gov/missions/noaa/ [Dickinson, 1995]).

5. Taiwanese Typhoon Counts

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

[31] Typhoons are the most serious natural disasters in Taiwan. They can produce strong winds, heavy rainfall leading to flooding, debris flows and landslides that result in loss of human life and damage to agriculture and property even if they do not make landfall [Wu and Kuo, 1999]. An example is Typhoon Morakot (August 2009) which took 675 lives (with 24 others missing) and caused U.S. $3.3 billion in damages [Chanson, 2010].

[32] Tu et al. [2009] describe a Bayesian change point analysis of a seasonal (June to October) series of typhoon counts in the vicinity of Taiwan (21°–26°N, 119°–125°E) for the period 1970 to 2006 (Figure 6). Typhoons were defined as events with a maximum surface wind speed greater than 34 kt (17.5 m s– 1). Their results indicated a change point around 2000, with rate parameters 3.3 yr−1 and 5.7 yr−1 for the epochs 1970–99 and 2000–06, respectively.

image

Figure 6. Local linear regression curve with hhdf= 3.18 (thick line), linear trend function (dashed line) and variability band (gray) for the null model (constant mean) for square-root transformed typhoon count series.

Download figure to PowerPoint

[33] Application of the Pettitt test to the count series suggests some evidence against the null hypothesis that the data are independent and identically distributed (bootstrap p-value = 0.07 for 10000 realizations). The value of the index of dispersion (variance-to-mean ratio) for the series is 0.84 which suggests that the data are under-dispersed. Our analysis of deviance comparing the homogeneous and nonhomogeneous Poisson models suggests that there is moderate evidence against the homogeneous model (bootstrapp-value = 0.021 for 10000 realizations). The selected values of the smoothing parameter for local linear regression arehaicc = 17.3, hcv = 9.87, and hdf = 3.18. We chose hdf because its corresponding cutoff frequency is less than 0.1 cycles per year (larger values of h will eliminate almost all of the structure in a series of only 37 years' duration), and restrict our attention to the range 2 ≤ h < 10 as it provides an appropriate degree of smoothing.

[34] Figure 7 displays significance traces for the null hypotheses of no trend (i.e., a constant mean), linear trend and no discontinuity in the transformed count series. It indicates: little to moderate evidence against the null hypothesis of no trend (0.032 < p-value < 0.22) throughout the considered range ofh; that there is little to no evidence against null hypothesis of a linear trend (0.20 < p-value < 0.62); and that there is no evidence of any discontinuities (0.61 <p-values < 0.99). On the basis of this, we fit the linear model(1)to the square-root transformed series using least squares. The estimated coefficients and their standard errors are inline image and inline image Diagnostic checks (not shown) indicate that the model assumptions are not unreasonable except that the observation for 2004 is influential (i.e., has a marked impact on the estimate of β1). The p-value for testing the hypothesisH0 : β1 = 0 is 0.029. Thus there is moderate evidence against the null hypothesis of no trend. Therefore we conclude that a linear trend function is a sufficient description of the underlying trend in the square-root transformed typhoon count series.

image

Figure 7. Significance traces for testing null hypotheses of no trend, linear trend and no discontinuity in the square-root transformed typhoon count series. Horizontal dashed line indicates thep-value 0.05.

Download figure to PowerPoint

6. Concluding Remarks

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

[35] Our intention has not been to provide definitive analyses of our three case studies. Instead, we have used these data sets to illustrate the statistical issues involved in trend estimation and change point detection. Our analysis framework is less restrictive than many others in allowing the data to speak for themselves, mainly because local linear regression can account simultaneously for smooth trends and discontinuities if these are present. These findings can be used to inform subsequent analyses of temporal changes in the underlying physical mechanisms, and the identification of policy responses that are appropriate for smoothly varying rather than abrupt climate change (and vice versa).

[36] We find that the mean level of the winter NAO index series exhibits a smooth but non-monotonic trend, with little evidence for the existence of abrupt changes. For the relative humidity series, we find evidence for three potential sharp discontinuities which appear to be related to marked changes in the observing system used for the NCEP-R1 project. For the square-root transformed typhoon count series, we find that: there is some to moderate evidence against the null hypothesis that the data are independently and identically distributed; a linear trend is a sufficient description of the underlying trend; and that there is no evidence for a change point. We reiterate, however, that the modeling framework used here has good power to detect genuine discontinuities when these are present in data that are not independently and identically distributed. Superficially, our findings appear to contradict those ofTomé and Miranda [2004], Fealy and Sweeney [2005] and Tu et al. [2009] in terms of the strength of evidence for the existence of abrupt shifts. However, the statistical test results are in fact all in agreement. In the analysis framework used by Tomé and Miranda [2004], any smooth trend that is not linear must be represented by a piecewise linear function (which, in fact, is a fair approximation to the smooth trend shown in Figure 1); Fealy and Sweeney [2005] use a largely ad hoc approach and make no formal attempt to quantify the statistical significance of their four claimed change points. In the analysis of Tu et al. [2009], even a smooth change in mean level must be represented by a step function with a single change point. Strictly speaking therefore, the test results reported by Tomé and Miranda [2004] and by Tu et al. [2009] provide evidence against the respective null hypotheses of linearity and a constant mean, but this cannot be used to claim the existence of change points. For further discussion of this general point about the interpretation of “significant” test results, see Wunsch [2010]; Bates et al. [2010]; Chandler and Scott [2011, section 2.4]; and Vincent et al. [2011]. From the perspective of change point detection, the main strength of the regression framework described herein is that the null hypothesis can accommodate constant mean levels, linear or smooth trends: rejection of this null hypothesis gives much greater confidence in the existence of one or more genuine change points.

Acknowledgments

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

[37] This work was carried out as part of the program Mathematical and Statistical Approaches to Climate Modeling and Prediction at the Isaac Newton Institute for Mathematical Sciences, Cambridge, UK. Financial support was provided by the Newton Institute.

References

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information

Supporting Information

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. Methods
  5. 3. Winter NAO Index
  6. 4. Relative Humidity Series
  7. 5. Taiwanese Typhoon Counts
  8. 6. Concluding Remarks
  9. Acknowledgments
  10. References
  11. Supporting Information
FilenameFormatSizeDescription
jgrd18119-sup-0001-t01.txtplain text document1KTab-delimited Table 1.
jgrd18119-sup-0002-t02.txtplain text document1KTab-delimited Table 2.

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.