Present address: Kinneret Limnological Laboratory, Israel Oceanographic and Limnological Research, PO Box 447, Migdal 14950, Israel

# A novel approach to detecting a regime shift in a lake ecosystem

Article first published online: 8 JAN 2010

DOI: 10.1111/j.2041-210X.2009.00006.x

© 2009 The Authors. Journal compilation © 2009 British Ecological Society

Additional Information

#### How to Cite

Gal, G. and Anderson, W. (2010), A novel approach to detecting a regime shift in a lake ecosystem. Methods in Ecology and Evolution, 1: 45–52. doi: 10.1111/j.2041-210X.2009.00006.x

Correspondence site: http://www.respond2articles.com/MEE/

#### Publication History

- Issue published online: 23 FEB 2010
- Article first published online: 8 JAN 2010
- Received 08 September 2009; accepted 23 November 2009 Handling Editor: Robert P. Frecklenton

### Keywords:

- free-knot spline;
- Lake Kinneret;
- Markov-switching vector autoregression;
- zooplankton

### Summary

- Top of page
- Summary
- Introduction
- Materials and methods
- Results and discussion
- Acknowledgements
- References
- Supporting Information

**1.** Certain classes of change in the characteristics of an ecosystem, labelled regime shifts, have been observed in marine and freshwater ecosystems world-wide. Few tools, however, have been offered to detect and identify regime shifts in time-series data.

**2.** We use a novel approach based on tools taken from the world of statistics, and econometrics to examine the occurrence of a regime shift in the predatory zooplankton population of Lake Kinneret, Israel. The tools are a free-knot spline mean function estimation method and a Markov-switching vector autoregression model.

**3.** Our approach detected, with high probability, the occurrence of a regime shift in the zooplankton population in the early to mid-1990s. This was in-line with expectations based on similar events observed in the lake.

**4.** The suggested approach is a step forward from existing approaches in that it does not require any pre-determent of threshold values but rather relies on a hidden underlying stochastic process that yields probabilities of regime shifts. Thus, it can therefore be applied without introducing any prior biases into the analysis. The approach is, therefore, an objective method in detecting the likely occurrence of a regime shift.

### Introduction

- Top of page
- Summary
- Introduction
- Materials and methods
- Results and discussion
- Acknowledgements
- References
- Supporting Information

Occasional, but dramatic shifts in the characteristics of an ecosystem, labelled regime shifts, have been observed in ecosystems world-wide. Carpenter (2003) defined regime shifts as: ‘a rapid modification of ecosystem organization and dynamics with prolonged consequences’. Regime shifts can, therefore, be considered as a move from one characteristic trend or state to a dramatically different trend or state (Scheffer *et al.* 2001). Such shifts have been documented for a range of ecosystems (Scheffer & Carpenter 2003 and references therein) including marine (Reid *et al.* 1998; Hare & Mantua 2000; Daskalov *et al.* 2007) and freshwater ecosystems (Carpenter, Ludwig, & Brock 1999). In lakes, the three common types of regime shifts include: eutrophication, changes to the fish populations, and shifts between a piscivore and large bodied grazers dominated system to one dominated by planktivores and small-bodied grazers (Carpenter 2003). Although perhaps, the most studied and document shift in lakes is the shift between phytoplankton and submerged macrophyte dominated ecosystems (Loverde-Oliveira *et al.* 2009; Zimmer *et al.* 2009). Anthropogenic factors such as increased nutrient loading, over-fishing, species invasions and habitat modifications are considered major culprits for shifts in freshwater ecosystems (Carpenter 2003; Genkai-Kato 2007; Roelke *et al.* 2007). In marine ecosystems, there is also evidence to suggest that altered levels of predation and competition may be playing a key role in reshaping trophic dynamics and regimes (Verity & Smetacek 1996; McCook 1999).

Recent work has focused on the theoretical background for regime shifts and has attempted to explain the mechanisms behind the shifts (Scheffer *et al.* 2001; Beisner, Haydon, & Cuddington 2003; Carpenter 2003; Scheffer & Carpenter 2003). The reasons, however, for these shifts are not always obvious and can be driven by large natural or human driven events or by gradual and incremental changes that have pushed the system beyond some sort of critical threshold level (Scheffer & Carpenter 2003; Carpenter & Lathrop 2008). The critical threshold is not, however, identical in both directions; the threshold at which a shift back can occur, following a perturbation, often differs from the original threshold. This difference in threshold levels is known as hysteresis (Scheffer *et al.* 2001). The degree of hysteresis following a shift can, however, vary greatly between ecosystems (Scheffer *et al.* 2001).

#### Regime identification

The identification of regime shifts in an ecosystem is not an easy task, especially in cases in which no abrupt change occurred to the variables in question (e.g. clear to turbid water). There may be theoretical reasons to believe that a regime shift occurred in the ecosystem, but that hypothesis of change may not be detected by an initial analysis of the observed data of the system. The detection of change could be complicated further if the system is very noisy. For example, a gradual change to the ecosystem behaviour, or a change in response to external forcing such as climatic events may have occurred (Beisner *et al.* 2003; Folke *et al.* 2004), and investigators may try to characterize these changes via such statistics as the mean and/or variance–covariance structure. These may not be sufficient statistical summaries, especially in a highly noisy system with subtle changes; thus the use of richer forms of statistical models is warranted.

Regime shifts, and especially the point in time in which they occur, are not always easily detectable and there have been only a limited number of cases in which attempts were made to develop and test methods for identifying regime shifts in ecological settings. Currently, there is no way to unambiguously detect a regime shift. At best, we can only make a probabilistic assessment of the likelihood of a regime shift. It is, however, important to differentiate between *subjective* and *objective* regime changes. A subjective view is one given by a few, but not all, of the experts as to whether a regime change occurred and may not be universally accepted by the relevant scientific community as a whole. In essence, this can be considered a qualitative view. An objective approach, often achieved through statistical modelling, removes the subjective viewpoint and offers a more detached view. In essence, this can be considered a quantitative view. When all the assumptions of a proposed regime shift model are met quantitatively, we can objectively conclude that a regime change has occurred. These conclusions, however, must be stated in light of the statistical and probabilistic nature of the model. Thus, we can talk about the probability or confidence of a certain regime occurring and the probability or confidence of a given regime not occurring through the statistical significance of the estimated regime shift parameter values. For example, we can construct a 95% confidence interval for a regime shift parameter value and this tells us that upon repeated experimentation, we expect the value to be within the interval 95% of the time and 5% of the time the parameter is not covered by the interval. In other words, 5% of the time, the inference will be wrong. This yields the confidence in the model. Taking the Bayesian approach, we can construct 95% credible intervals and ask what is the probability of observing a regime shift parameter value? Therefore, to ask the question ‘did a regime shift actually occur?’ in this framework is not correct. The correct form of a question would be ‘what is the probability or confidence that a regime shift occurred?’

Carpenter & Pace (1997) attempted to use three simple statistical and mathematical models to evaluate the existence of alternative stable states, however, they reached only inconclusive results. Carpenter (2003) used a simple mathematical model to evaluate alternative equilibria in a simulated time series and concludes that the ability to detect alternative equilibria, based on statistical approaches alone, will be limited for most ecological time series. Ives *et al.* (2003) took a slightly different approach by applying single order vector autoregressive models [defined as MAR(1) by the authors though considered VAR(1) in the statistics and econometrics literature] to ecological time-series data to estimate planktonic community stability in three lakes but did not clearly provide a means for defining the occurrence of a regime shift. Carpenter & Lathrop (2008) estimate the probabilistic threshold for eutrophication using the joint density of the parameters of a deterministic differential equation via a Bayesian approach. While all of the above cases represent attempts to quantify regime shifts with simple models, they offer no theoretical or data-driven justification for why these models should be used. There is a need, therefore, to develop and implement methods that will allow statistical, or objective, detection of regime shifts based on available time-series data.

The goal of this study is to develop and test a novel approach to detect regime shifts in ecological time-series data by applying tools borrowed from multiple disciplines. In contrast to other regime shift modelling efforts, we take a more general approach in modelling regime changes not as a deterministic process, but as an unobservable or latent underlying stochastic process. We use the proposed method to evaluate whether a regime shift occurred, in probabilistic terms, in the Lake Kinneret zooplankton population during the early to mid-1990s.

### Materials and methods

- Top of page
- Summary
- Introduction
- Materials and methods
- Results and discussion
- Acknowledgements
- References
- Supporting Information

#### Lake Kinneret data

The long records of data from Lake Kinneret in addition to the recent changes and major perturbations that have occurred in the lake make it an ideal ecosystem to study regime shifts. The lake has been studied since 1969 and a wide range of physical, biological and chemical variables have been monitored routinely ever since (Berman *et al.* 1995). The lake exhibited stability in its characteristics over a range of trophic levels since the initiation of the monitoring programme through to the early, mid-1990s when large changes occurred (mid-1990s event, MNE, hereafter). The most prominent change was a shift in the phytoplankton annual succession pattern. The succession pattern shifted from a stable dinoflagellate dominated ecosystem to an unpredictable ecosystem in which annual algal blooms switch between dinoflagellates to various cyanobacteria species. The succession pattern, in the lake, can be divided into two major periods; 1969–1993 and 1994–current as described by Zohary (2004). The reasons for the changes are not clear, though a number of processes have been hypothesized (Gophen *et al.* 1999; Zohary 2004). Recently, Roelke *et al.* (2007) argued that the change in the algal population represented a shift between alternate stable states.

The zooplankton time series used in this study is based on the routine monitoring of zooplankton in the lake. Lake Kinneret zooplankton have been sampled fortnightly on a routine basis at a mid-lake station since 1969 (Gophen, Serruya, & Threlkeld 1990) and the sampling methodology has changed very little since it was initiated. We extracted the time-series data from the Lake Kinneret data base (Kinneret Limnological Laboratory 2001). The samples counted represent a mix of samples collected at discrete depths. Samples were collected at 0, 1, 2, 3, 5, 7, 10, 15, 20, 30 and 40 m depths during the non-stratified periods (typically January to March). During stratified periods, samples were collected to the sampling depth closest to the thermocline and, in addition, samples were collected from 1 m above and below the thermocline and from the mid-depth of the thermocline. All samples collected were combined into a large container and stirred. Once well mixed, a sample of 0·8–1·0 L was collected from the container and counted, thus representing a pseudo-integrated sample of the water column. Complete samples were counted to the species level. All count data were stored in the Lake Kinneret data base and converted to areal densities (number m^{−2} according to sampling depth). We utilized all samples for the period January 1970 through December 2006. The fortnightly data were averaged to create mean monthly values. We use natural log-transformed data (of the mean monthly areal densities +1) for all calculations. In this study, we focus on the changes in density of the adult stages of the two predatory cyclopid species in the lake, thus studying the changes that occurred to the predatory zooplankton population in the lake.

#### Stationary time series

Weak or second order stationarity is a very important concept in time-series analysis and it is essential for the statistical analysis of time-series data. Recall the definition of a weak or second order stationarity process *Y*(*t*) is:

- (eqn 1)

- (eqn 2)

where *E*[·] and *C*[·] are the expectation and covariance operators, respectively. Through the principle of superposition, time-series data is a composition of a trend component, a seasonal component and a stationary component. The trend and seasonal components are typically not stationary, so, in general, time-series data are not stationary. Therefore, to use time-series analysis tools, the data must be second order stationary. There are several statistical tests for stationary data. See Shumway & Stoffer (2006) for further details.

There are several ways to induce second-order stationarity. One way is to difference the data. By the differencing method, the data are transformed into a new and stationary data set. That is, the data are no longer in the observed unit of measure, but measured in relative units of measure. For example, if we were to sample zooplankton density data in units of the number of individuals per litre, the differenced data would be in units of the change in number of individuals per litre. This is a different data set than the one originally sampled.

Another way to induce stationarity is to estimate the trend and seasonal components and remove them from the observed time series. This yields a stationary data set via the principle of superposition. Using this approach, the data are kept in the original units as sampled. This is the approach we use in this article. We estimate the trend via free-knot splines and remove the trend from the data. The resulting data are stationary and we can apply statistical procedures for analysis. Details on the free-knot spline approach are given below.

Very often, the sampled data points are noisy enough to mask patterns of interest such as trends and seasonality in the series. This leads to difficulties when attempting to model the trend in the data. Thus, researchers turn to various types of smoothing such as data transformations and using lower sampling frequencies (Shumway & Stoffer 2006). Popular smoothing methods are in the form of filters, such as moving averages, or in the form of functional analysis, such as spline smoothing (Shumway & Stoffer 2006). The idea behind smoothing or filtering the noisy data is to get at the functional form of the trend, or seasonality. However, these smoothing techniques do not always detect subtle, but important, features of the data because they tend to over-smooth the data because of the smoothing parameter estimation method. In many instances in the functional analysis approach, the focus is on individual smoothing of the sample curve followed by cross-sectional averaging and covariance computation (Ruppert, Wand, & Carroll 2003). In the approach used in this article, we by-pass the over-smoothing issue by applying a free-knot spline estimator (Gervini 2006) of

- (eqn 3)

where *E*[·] denotes the expectation operator on the stochastic process (time series) *X*(*t*). The result is a piecewise continuous function where the ‘pieces’ are joint at the knots. This approach very often produces better estimates of the mean and covariance of the stochastic process, which in this case, is the trend we would like to estimate.

#### Free-knot basis splines

Recall that splines are piecewise polynomial curves that are differentiable up to a prescribed order, where the polynomial pieces are joined at the so-called knots. Usually the knots of the spline are specified and fixed *a priori*. A special class of splines is the basis splines (B-splines). These arise from the divided differences of Green’s function, which are piecewise polynomials that can be used to construct bases for spline spaces (see Shumacher 1981 for further details). In our approach, we do not specify *a priori* or use fixed-knot placement but take a free-knot approach in which we determine the optimal free-knot vector thus allowing the data to ‘speak for itself’ as to the amount and the placement of knots.

To estimate the mean function (the trend) *μ*(*t*) via free-knot splines, we assumed that the data are discrete realizations from the curve (or a family of curves):

- (eqn 4)

where *X*_{i}(*t*_{j}) is a stochastic process, ε_{ij} are independent and identically distributed (i.i.d.) random errors. The 1 ≤ *i *≤ *n* denotes the *i*th curve (in the multivariate case) and 1 ≤ *j *≤ *m* denotes the *j*th observation indexed by time. Furthermore, we assumed that the samples were equally spaced, but this assumption can be relaxed. Hence, we posit that the data follow the model

- (eqn 5)

and estimate *μ* with B-splines. The variance operator is denoted by *V*[·].

In our approach to the estimation of *μ*(t) = *E*[*X*(*t*)] with B-splines, we allow the knots to become free parameters and not fixed as in the usual approach. More formally, the idea is that given an initial vector of knots τ in the set [*a*, *b*], we can transform the knots via a Jupp transformation κ = *J*(τ), where

- (eqn 6)

for *l *=* *1, … , *p* and the end conditions *τ*_{0} = *a*, *τ*_{p+1} = *b*. The Jupp transformation maps the constrained, increasing order knot vector τ onto an unconstrained, unordered vector κ. Let *B*(*t*, κ) be the vector of base functions corresponding to κ and let *B*(κ) be the matrix whose *j*th row is *B*(*t*_{j}, κ)^{T}, where T denotes the transpose of a matrix. From the definition of B-splines, set , where denotes the vector of sample control points. The objective function is the minimization of the quadratic error between the control points and the Jupp knots. That is,

- (eqn 7)

where *R*^{r+p} × *R*^{p} denotes the dimension of the Euclidian space of real numbers and *r* is the order of the spline with degree *r *− 1. When κ is given and the optimal control point vector is given by

- (eqn 8)

so our minimization problem becomes

- (eqn 9)

which is a nonlinear optimization problem. The objective function is minimized when we have found , the optimal knot vector.

The selection of the optimal knot order *p* has been discussed in the statistics and functional data analysis literature extensively (Hastie, Tibshirani, & Friedman 2001). The most commonly used approach is to find *p* that minimizes an estimate of the mean average squared error . Usually, is estimated by cross-validation, which is very computationally expensive. Here, we use generalized cross-validation (GCV), which is defined in terms of the average squared error as,

- (eqn 10)

where and *d* = 2*p + r* is the effective number of parameters. The strategy carried out is to use a relatively large initial guess of *p**, compute the GCV criterion for all intermediate knot vectors and choose the that minimizes the GCV.

#### Time series analysis

Large changes in time series, whether abrupt or gradual, such as those defined as regime shifts occur in many forms of time series such as ecological, financial and economic data. For example, intense agricultural activity in a lake watershed, previously devoid of agriculture, will most likely lead to a sharp increase in nutrient loading followed by rapid eutrophication of the lake and, as a result, a shift from clear to relatively turbid water. A natural question is how to model these types of changes. More formally, we can model the eutrophication process as an autoregressive process of the form

- (eqn 11)

before agriculture development, and as

- (eqn 12)

after agricultural activities start, where μ_{1} > μ_{2}. It is clear that this simple autoregressive model depends on the state of the regime through the constant parameters μ_{1} and μ_{2}. This is the regime dependent model. Note that the regimes can be deterministic, such as seasonal effects on lake algal biomass, or they can be stochastic, such as the multiannual variation in chlorophyll concentrations in a lake over time. We focus our efforts on the stochastic regime changes.

Modelling nonlinearities in time-series data is an active area of research. One branch of this area is to use piecewise linear models to capture nonlinear effects (Franses & Van Dijk 2000). This approach can be used to model structural breaks in the data generating process. An example is given by our hypothetical watershed development above. While structural breaks or regime changes seem natural to many scientists, there is little established theory suggesting ways of modelling these regime changes quantitatively. One way is to specify the structural breaks *a priori*, and then apply linear models around the regimes. Examples of this approach are the class of self-exciting autoregressive (SETAR) models, and the class of smooth transition autoregressive (STAR) models. A downside to using these models is that the regime changes are exogenously specified and therefore, a *subjective* approach. Since regimes are typically unobserved, an alternative approach can be taken in which they are modelled by a stationary, ergodic Markov chain (see below). Thus, we are able to *objectively* classify the likelihood or probability of the system being in a specific regime.

Our general approach is to model regime shifts not as a deterministic process, but rather driven by an unobservable or latent underlying stochastic process. When using stochastic regime shift models, it is assumed that the regimes are latent states and cannot be observed. Since the regimes cannot be directly observed, we can only assign probabilities of the occurrence of the different regimes. To illustrate the ideas above and for the ease of exposition, we consider a simple univariate two regime AR(1) process, such as Secchi depth before and after a key event as used above. More mathematically, let *S*_{t} be the unobserved stochastic process and in the case of a two regime AR(1), we have

- (eqn 13)

where *s*(*t*) = 1 for regime 1 and *s*(*t*) = 2 for regime 2 and ε_{t} is i.i.d. *N*(0, σ^{2}). This model allows for two states: regime 1 and regime 2. The regime *s*(*t*) is modelled as the outcome of an unobserved Markov chain with *s*(*t*) independent of the error process ε_{t}. The use of Markov chains as the hidden process generating the regimes offers tremendous flexibility in modelling. We can model the permanent change in regime 2 by specifying regime 2 as an absorbing (a property of a Markov Chain) state. Note the use of a Markov chain implies that current regime *s*(*t*) depends only on the prior regime *s*(*t* − 1). That is,

- (eqn 14)

With this property, the probability of moving from one regime to another is given (in the two state model) by:

- (eqn 15)

These transition probabilities are usually written in the form of a transition matrix

- (eqn 16)

Using the theory of ergodic Markov chains, we can derive analytic probabilities that are unconditional probabilities that the process is in each of the regimes. More formally with these ergodic properties, the probability for the system to be in regime 1 (*s*(*t*) = 1) and regime 2 (*s*(*t*) = 1) in the two state model, is given by:

- (eqn 17)

As stated above, the two regime AR(1) process is rather simplistic. We can expand the basic idea outlined above in several ways. One way is to allow for more than two regimes (*s*(*t*) > 2). Other ways to enrich the Markov-switching autoregressive processes is to allow time-dependent parameters and allow multivariate processes as well as allow non-constant (heteroskedastic) variance. Again, these parameters depend on the state of the system. These generalized stochastic processes are called Markov-switching vector autoregression models and are denoted by MS(*m*)VAR(*p*), where *m* denotes the number of regimes in the Markov process and *p* denotes the number of lags in the autoregressive process. We use these types of processes to model the data. See Krolzig (1997) for further details and Appendix S1 (Supporting information) for the full model and estimation steps.

Of note, there is a similar branch of regime analysis in the mathematics, statistics and engineering literature under the name change point analysis. Regime change is a nomenclature mainly used in finance, economics and ecology. In essence, they are the same concepts, where researchers are looking for changes in the state or system. There is a countless number of approaches to this problem. These system changes are usually manifested in the parameters of the system, e.g. a change in the level (a type of change point or a type of regime change) or a change in the variance. Our MS(*m*)VAR(*p*) approach simply assumes the number and location of the change points is unknown and follows a Markov chain. From this modelling structure, we can assign the probability of a change point or regime change that is reflected through the parameters, such as level, variance and autocorrelation. The state-space modelling approach to change points, e.g. the MS(*m*)VAR(*p*) approach, was introduced by Harrison & Stevens (1975).

### Results and discussion

- Top of page
- Summary
- Introduction
- Materials and methods
- Results and discussion
- Acknowledgements
- References
- Supporting Information

The natural log-transformed predatory zooplankton data are plotted in Fig. 1. To estimate the trend in the noisy data, a cubic smoothing spline and a cubic free-knot B-spline approach were applied to predatory zooplankton data. A smoothing parameter of *p* = 0·9 is used for the smoothing spline. The GCV optimized knot vector for the free-knot splines is = (15·5, 22·8, 222·9, 278·3, 278·9, 278·9, 280·1, 331·4, 386·4), where the values represent the number of months since January 1970. The two approaches give somewhat similar patterns but with notable differences occurring during periods of large changes in the data. The free-knot spline captured the large changes to the log predator density in the early 1970s, the mid-1990s and the early 2000s (Fig. 1). This is represented in the optimized knot vector, where there were two knots positioned at 15 and 22 (corresponding to the early 1970s), four positioned around 280 (corresponding to the year 1993) and two positioned at the end of the data series. Compare this with the cubic smoothing spline in the figure, which largely smoothed over these areas in the data. Although not a formal statistical test, but rather a result of the optimization of knot placements, it is interesting to note that the locations of the nine free-knots correspond with regions of large changes in the trend over the course of the time series. A higher clustering of knots indicates larger variability in the data for more cubic polynomials are joined together to yield a better fit of the trend. As noted above, four of the nine knots were placed within a period of several months during 1993.

A closer examination of the differences in the trend estimation between the two spline methods, especially for the 1990–2006 period, illustrates the benefits of the free-knot approach. The smoothing cubic spline method gives a very smooth trend with relatively minor perturbations in the zooplankton density occurring mainly in 1992–1994 and again in 1999. The free-knot spline, on the other hand, shows major perturbations occurring in 1993, 2001 and again in 2004–2005. The differences between the perturbations in the 1990s and in 2004–2005 are especially noteworthy with the transformed density values rapidly falling by over 50% in 1993 as seen in the free-knot spline as well. In 2004–2005, the trend observed in the raw data is well captured by the free-knot approach but not by the smoothing spline approach.

Various classes of MS(*m*)VAR(*p*) models (see Appendix S1 for details) were fitted to the detrended data. Stationarity tests such as unit root tests (Franses & Van Dijk 2000) were performed on the detrended data and indicated data stationarity. Unfortunately, standard asymptotic distribution theory cannot be used for tests concerning the number of states of the Markov chain. This is because of the existence of nuisance parameters under the null hypothesis, which leads to identifiability problems. In light of this, we used log-likelihood statistics for regime model selection as suggested by Krolzig (1997). With the number of regimes fixed, standard log-likelihood ratio tests were used for sub-model testing. The final model suggested by various model diagnostic procedures was a MSIH(4)AR(4), where MSIH denotes we are modelling the intercept (*I*) with non-constant variance (*H*) with four regimes and a autoregressive process of four lags. The fitted model and associated statistics are as follows:

where and *ξ* is the state or regime vector. Variance estimates within the four regimes (i.e. change points) are , , , . The *p* values and regime intercept values (i.e. change points) are given in Table 1. Standard errors are given in parenthesis. The parameter estimation was done using the EM algorithm (a maximum-likelihood estimation technique) (Dempster, Laird, & Rubin 1977). Note that the above model is a vector auto regression with four lags, and resides with various probabilities in four different states or regimes as described by the parameter estimates.

Initial regime | Target regime | ||||
---|---|---|---|---|---|

1 | 2 | 3 | 4 | ||

^{}For example, the probability of shifting from regime 2 to regime 4 is 0·5043. That is, *Pr*{*s*(*t*) = 4|*s*(*t*–1 = 2)} = 0·5043. Regime intercept values are given by . Standard errors are given in parenthesis
| |||||

1 | 0·4122 | 0·2068 | 0·0125 | 0·3685 | |

2 | 0·0978 | 0·3952 | 0·0027 | 0·5043 | |

3 | 0·0185 | 0·0003 | 0·9806 | 0·0007 | |

4 | 0·1711 | 0·2147 | 0·0185 | 0·5957 |

The probabilities of the system to reside in one of the four statistically significant regimes are plotted over time in Fig. 2. There are some quite outstanding features in this figure, especially in light of the Markov chain transition matrix given in Table 1. The top four panels of Fig. 2 are plots of the four regime probabilities, regime 1 through regime 4, and the bottom panel is a plot of the stationary (detrended) time series. It is important to keep in mind that we are attaching probabilities of the system (natural log density of predatory zooplankton) to lie in each of the states or regimes and *not* stating that the system is in regime 1 or 2, etc. Probabilities must be attached to each regime.

For roughly two-thirds of the time series (until *c.* 1992), the system of zooplankton tends to stay in regimes 2 or 4, with relatively high probabilities. This sequence of rapid changes in the regimes is most likely because of the system being at the cusp of regimes 2 and 4. There is also an occasional stay in regime 1, but with lower probability. Regime 3 is rarely visited as indicated by the low probability. This can be clearly seen in the transition matrix (Table 1). The probability of transitioning from regime 2 to regime 2 is 0·3952, while the probability of transitioning from regime 2 to regime 4 is 0·5043 and the probability of transitioning from regime 2 to regime 3 is only 0·0027. The probability of transitioning from 2 to 1 (0·2068) is half of the probability of staying in 2. Another outstanding feature of the regime probability plots is that from 1993, the Markov chain chiefly stays in regime 3. The probability of staying in regime 3 from regime 3 is 0·9806, which is nearly an absorbing state (a property of Markov chains). Only towards the end of the sample does the data shift temporarily out of regime 3. Given the short period of time between this event and the end of the data series we cannot draw firm conclusions.

The method of estimating trends via a free-knot spline and modelling possible regime shifts of the detrended data via unobservable underlying stochastic process, as outlined above, clearly identifies a regime shift occurring in the Lake Kinneret predatory zooplankton population. The timing of the shift coincides with other events that occurred in the lake and is in sync with expectations given our knowledge of the ecosystem in question. Previous reports identified a major change in the phytoplankton annual succession pattern (Zohary 2004) and a shift between alternatives states occurring in the phytoplankton population at the same period of time (Roelke *et al.* 2007). While the changes in the phytoplankton population, as a result of the shift between alternative states, were noticeable, changes to the zooplankton population were not as clear with the exception of a low minimum in zooplankton biomass occurring in 1992 and 1993 (Fig. 1). The results suggest that there were changes to the population beyond a brief period of very low densities. Interestingly, a repeat of the process that lead to the minima in zooplankton densities (Ostrovsky & Walline 1999) occurred in 2003 but did not result in an as noticeable impact on the zooplankton population. This would be expected if the lake was residing in a different regime or an alternate stable state (Roelke *et al.* 2007) from that prior to the 1990s event, thus indirectly supporting our results.

Only a limited number of attempts, which were not always successful, have been made to model regime shifts or similar events such as movement between alternative stable states in aquatic systems. Carpenter (2003), for example, concluded that statistical approaches alone are limited in their ability to detect alternative equilibria in most ecological time series based on the statistical methods he used on a simulated time-series data set. Indeed a number of statistical approach used by Carpenter & Pace (1997) to detect alternative stable states met limited success. This, however, may have been a result of the use of non-stationary data. Additionally, they applied a deterministic threshold that requires an a priori selection of the threshold thus introducing bias into the analysis and limiting its wider applicability. Ives *et al.* (2003) applied a multivariate first-order autoregressive model to study the stability of ecological time-series data, however, the issue of detecting regime shifts via these class autoregressive models is not addressed.

One of the main benefits of our approach is the utilization of methods and tools borrowed from other fields unrelated to limnology thus taking advantage of progress made elsewhere in analyzing time-series data set. Furthermore, we address the difficulty of objectively defining a regime shift in a given population based on a time series. In this study, we utilized tools to determine if the predatory zooplankton of Lake Kinneret underwent a regime shift as an example of our approach. Here, we provide a means to quantitatively evaluate the occurrence of regime shift events in a time-series data set. It offers a clear and objective method for evaluating the existence of regime shifts that are occurring with increasing likelihood as the result of human activity (Carpenter 2003; Folke *et al.* 2004).

### Acknowledgements

- Top of page
- Summary
- Introduction
- Materials and methods
- Results and discussion
- Acknowledgements
- References
- Supporting Information

We are grateful to Beatrix Beisner and Lars Rudstam for helpful comments on an earlier draft of the manuscript. We thank the anonymous reviewers for their constructive review. We thank the Kinneret Limnological Laboratory field crew and zooplankton technicians over the years for their dedicated efforts. We also thank Miki Schlicter for data base support.

### References

- Top of page
- Summary
- Introduction
- Materials and methods
- Results and discussion
- Acknowledgements
- References
- Supporting Information

- 2003) Alternative stable states in ecology. Frontiers in Ecology and the Environment, 1, 376–382. , & (
- 1995) Primary production and phytoplankton in Lake Kinneret: a long-term record (1972–1993). Limnology and Oceanography, 40, 1064–1076. , , , , , & (
- 2003) Regime Shifts in Lake Ecosystems: Pattern and Variation. International Ecology Institute, Oldendorf/Luhe. (
- 2008) Probabilistic estimate of a threshold for eutrophication. Ecosystems, 11, 601–613. & (
- 1997) Dystrophy and eutrophy in lake ecosystems: implications of fluctuating inputs. Oikos, 78, 3–14. & (
- 1999) Management of eutrophication for lakes subject to potentially irreversible change. Ecological Applications, 9, 751–771. , & (
- 2007) Trophic cascades triggered by overfishing reveal possible mechanisms of ecosystem regime shifts. Proceedings of the National Academy of Sciences, 104, 10518–10523. , , & (
- 1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39, 1–38. , & (
- 2004) Regime shifts, resilience, and biodiversity in ecosystem management. Annual Review of Ecology, Evolution, and Systematics, 35, 557–581. , , , , , & (
- 2000) Nonlinear Time Series Models in Empirical Finance. Cambridge University Press, Cambridge. & (
- 2007) Regime shifts: catastrophic responses of ecosystems to human impacts. Ecological Research, 22, 214–219. (
- 2006) Free-knot spline smoothing for functional data. Journal of the Royal Statistical Society (Ser.B), 68, 671–687. (
- 1990) Long term patterns in nutrients phytoplankton and zooplankton of Lake Kinneret Israel and future predictions for ecosystem structure. Archiv fur Hydrobiologie, 118, 449–460. , & (
- 1999) Nitrogen deficiency, phosphorus sufficiency, and the invasion of Lake Kinneret, Israel, by the N-2-fixing cyanobacterium
*Aphanizomenon ovalisporum*. Aquatic Sciences, 61, 293–306. , , & ( - 2000) Empirical evidence for North Pacific regime shifts in 1977 and 1989. Progress in Oceanography, 47, 103–146. & (
- 1975) Bayesian forecasting (with discussion). Journal of the Royal Statistical Society, Series B (Methodological), 38, 205–247. & (
- 2001) The Elements of Statistical Learning. Springer-Verlag, New York, NY, USA. , & (
- 2003) Estimating community stability and ecological interactions from time-series data. Ecological Monographs, 73, 301–330. , , & (
- Kinneret Limnological Laboratory (2001) Lake Kinneret Database: Limnological Data on Lake Kinneret Since 1968. Israel Oceanographic and Limnological Research, Yigal Allon Kinneret Limnological Laboratory, Tiberias.
- 1997) Markov-switching Vector Autoregressions: Modelling, Statistical Inference, and Application to Business Cycle Analysis. Springer, Berlin. (
- 2009) Hydrology-driven regime shifts in a shallow tropical lake. Ecosystems, 12, 807–819. , , & (
- 1999) Macroalgae, nutrients and phase shifts on coral reefs: scientific issues and management consequences for the Great Barrier Reef. Coral Reefs, 18, 357–367. (
- 1999) Growth and production of the dominant pelagic fish,
*Acanthobrama terraesanctae*, in subtropical Lake Kinneret, Israel. Journal of Fish Biology, 54, 18–32. & ( - 1998) Phytoplankton change in the North Atlantic. Nature, 391, 546–546. , , & (
- 2007) Alternative states in the phytoplankton of Lake Kinneret, Israel (Sea of Galilee). Freshwater Biology, 52, 399–411. , , & (
- 2003) Semiparametric Regression. Cambridge University Press, Cambridge, U.K. , & (
- 2003) Catastrophic regime shifts in ecosystems: linking theory to observation. Trends in Ecology & Evolution, 18, 648–656. & (
- 2001) Catastrophic shifts in ecosystems. Nature, 413, 591–596. , , , & (
- 1981) Spline Functions: Basic Theory. John Wiley & Sons, Inc., New York, NY, USA. (
- 2006) Time Series Analysis and its Applications. Springer Texts in Statistics, 575 pp. Springer, New York, NY, USA. & (
- 1996) Organism life cycles, predation, and the structure of marine pelagic ecosystems. Marine Ecology Progress Series, 130, 277–293. & (
- 2009) Thresholds and stability of alternative regimes in shallow prairie–parkland lakes of central North America. Ecosystems, 12, 843–852. , , & (
- 2004) Changes to the phytoplankton assemblage of Lake Kinneret after decades of a predictable, repetitive pattern. Freshwater Biology, 49, 1355–1371. (

### Supporting Information

- Top of page
- Summary
- Introduction
- Materials and methods
- Results and discussion
- Acknowledgements
- References
- Supporting Information

**Appendix S1**. A detailed description of MS(*m*)Var(*p*)models with heteroskedastic variance.

As a service to our authors and readers, this journal provides supporting information supplied by the authors. Such materials may be re-organized for online delivery, but are not copy-edited or typeset. Technical support issues arising from supporting information (other than missing files) should be addressed to the authors.

Filename | Format | Size | Description |
---|---|---|---|

MEE3_6_sm_Appendix_S1.doc | 755K | Supporting info item |

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.