**Statement of authorship:** GIH had primary responsibility for data analysis, writing code, and preparing the initial manuscript; GAF provided support and discussion throughout the project, and contributed substantially to its final form.

LETTER

# Sampling stochasticity leads to overestimation of extinction risk in population viability analysis

Version of Record online: 7 JAN 2013

DOI: 10.1111/j.1755-263X.2012.00305.x

©2013 Wiley Periodicals, Inc.

Additional Information

#### How to Cite

Herrick, G. I. and Fox, G. A. (2013), Sampling stochasticity leads to overestimation of extinction risk in population viability analysis. Conservation Letters, 6: 238–246. doi: 10.1111/j.1755-263X.2012.00305.x

**Editor**Richard Zabel

#### Publication History

- Issue online: 13 AUG 2013
- Version of Record online: 7 JAN 2013
- Accepted manuscript online: 15 NOV 2012 08:10PM EST
- Manuscript Accepted: 30 OCT 2012
- Manuscript Received: 18 MAY 2012

#### Funded by

- G. A. Fox. Grant Number: DEB-0614468

- Abstract
- Article
- References
- Cited By

### Keywords:

- PVA;
- stochastic demography;
- stochastic exponential growth;
- mark-recapture;
- diffusion approximation;
- bias estimation;
- sampling variance;
- extinction risk

### Abstract

Which method should be used for estimating extinction risk? We present four separate estimates of extinction risk for the threatened pine lily (*Lilium catesbaei* Walter), based on two methods of estimating abundance (direct abundance counts and Jolly–Seber abundance estimates) and two methods of estimating extinction risk (direct simulation of the stochastic exponential growth (SEG) model, and the diffusion approximation). We compare the accuracy of these four combinations with a simulated data set where simulated-true population abundance and extinction risk is known. The Jolly–Seber method of abundance estimation in combination with direct estimation of extinction risk is the least biased combination of the four methods tested. We conclude that Jolly–Seber (or other mark-recapture) estimates should be used in combination with direct simulation of the SEG, when sampling error is expected. For the pine lily, we conclude that risk of extinction is low in the population studied.

### Introduction

Population viability analyses (PVAs) provide estimates of extinction risk and its components. PVAs based on count data involve two major steps. One first estimates population abundance, and then one uses a model to produce a risk estimate. Figure 1 provides a conceptual model of these data sources and models. The stochastic exponential growth model (SEG) uses annual population estimates to estimate the mean rate of population change (μ) and its variance (σ^{2}). Recently, Kendall (2009) showed that the diffusion approximation (an analytical method) to the SEG model (Dennis *et al*. 1991) overestimates extinction risk for count-based PVA (Morris & Doak 2002), relative to direct stochastic simulation of the model. Here, we compare risk estimates in a population of pine lilies (*Lilium catesbaei* Walter) based on two sources of abundance estimates, the direct enumeration and Jolly–Seber (hereafter, JS) estimates. JS is a mark-recapture method that takes into account variability due to low probability of counting individuals during any sampling event; direct enumeration is a simple count of individuals. We expect use of JS to reduce bias in PVA.

PVAs are subject to three sorts of uncertainty due to three types of variability: sampling error, demographic stochasticity, and environmental stochasticity. Measurement error in annual abundance estimates (*N _{t}*) necessarily results in overestimates of σ

^{2}. The SEG model treats all stochasticity as environmental stochasticity; demographic and sampling stochasticity thus inflate SEG estimates of environmental stochasticity and consequently of risk. Efforts have been made to reduce the effect of measurement error when only the time series of counts is available, by separately estimating sampling and environmental variability (Holmes & Fagan 2002; Lindley 2003; Staples

*et al*. 2004; Buonaccorsi & Staudenmayer 2009) but this comes at cost of higher uncertainty in the variance estimates.

An alternative approach to reducing bias in PVA is to improve abundance estimates through reducing sampling error in the abundance estimates by using mark-recapture methods, including the JS and Pradel methods. JS uses calculations that can be done by hand using sighting histories of individuals to arrive at estimates of abundance (Jolly 1965; Seber 1965; Schwarz & Seber 1999; Schwarz 2001). JS abundance estimates can be used in place of direct enumeration to reduce bias in calculations of annual population trends (*N _{t}*/

*N*) (Bryja

_{t−1}*et al*. 2001). The Pradel (1996) method is more powerful but more difficult computationally (requiring function minimization), while JS can be done on a hand calculator. By using both JS and direct enumeration, we examine the effect of sampling stochasticity on estimates of extinction risk.

One purpose of this study is to compare bias in μ and *σ*^{2}, from two kinds of abundance estimates: direct enumeration and JS. We approach this problem using individual-based simulations. These simulations produce three sets of annual abundances: the simulated-true values, the results of direct enumeration from simulated sampling, and JS estimates based on simulated sighting histories. These three estimates of abundance each produce an estimate of μ and *σ ^{2}*; we use them to parameterize both a diffusion approximation PVA and direct simulations of the SEG in order to compare performance of JS and direct enumerations on both forms of risk estimation.

Our simulation approach provides known, underlying population sizes prior to observation—data we could never collect in the real world. Thus, we can parameterize the SEG model with simulated-true population sizes, in addition to direct enumeration and JS estimates. Our results provide insight into the utility of mark-recapture methods for count-based PVA using the diffusion approximation and direct simulation of the SEG.

Our focal species, *Lilium catesbaei* (Liliaceae; pine lily) is native to Florida pine flatwoods (Myers & Ewel 1990). Flowering in this species increases following fire and mechanical clearing of underbrush (Huffman & Werner 2000; Sommers *et al*. 2011). Sighting of lilies is obscured because of their life history, which includes dormancy (sometimes for multiple years, as is common in many geophytes) and irregular flowering. This species is considered threatened in Florida, but these characteristics make monitoring and risk assessment difficult. Our use of *L. catesbaei* data allows our study to serve a second goal: examining the population viability of a population of this species.

### Materials and methods

#### Lily natural history and habitat

In 1995 the Pinellas County (Florida) Environmental Lands Division began monitoring the population of *L. catesbaei* at Brooker Creek Preserve, a 3,400 ha preserve in west-central Florida. Over the course of 8 years, over 1,500 individuals were identified with exhaustive searches in five natural flatwoods and several specially managed sites. They were marked and mapped with high-precision GPS, and recensused annually. In the second sampling year, no new individuals were added, so this year was dropped from our analyses.

We selected one of these sites, in approximately 5 ha of open flatwoods, bordered by a cypress swamp and power-line easements. Vegetation included a sparse canopy of slash pine (*Pinus elliottii var. elliottii*) and a dense shrub layer of saw palmetto (*Serenoa repens*), gallberry (*Ilex glabra*), staggerbush (*Lyonia fruticosa*), dwarf live oak (*Quercus minima*), wax myrtle (*Myrica cerifera*) and creeping huckleberry (*Gaylussacia nana*). Habitat requirements for *L. catesbaei* are explored by Sommers *et al*. (2011).

The pine lily is a perennial geophyte, with a below-ground bulb. Above-ground growth is present only for a portion of the year, and is typically absent during winter. Lilies are difficult to identify unless they are flowering. Thus, monitoring focused on flowering individuals. Simulations mimicked this natural history and consequent restraints on sampling efficacy.

Capture histories were created from annual censuses and tabulated into a method B table for JS estimation of abundance, survival, and reproduction. We estimated μ and *σ ^{2}* for the SEG model from the abundance estimates using linear regression (Dennis

*et al*. 1991, Morris & Doak, 2002), which allowed for missing years and consequent inequality of inter-sample times. We also estimated abundance, annual population growth rates, and sighting probabilities using the Pradel method in program MARK (Pradel 1996, White & Burnham 1999).

#### Simulation

Simulations began with one of three initial population sizes (*N _{0}* = 50, 100, 200). Each individual was given a lifespan drawn randomly from a geometric distribution with a mortality rate of 10%. This resulted in an average lifespan of 9 years, consistent with our field data. Each individual had a chance of reproducing represented by an annual random draw from a Poisson distribution with a birth rate of 10%. This set our expected population growth rate to zero, where future population size is most sensitive to small perturbations. Each initial population size had a corresponding quasi-extinction threshold 50 individuals fewer than

*N*, or 50% of

_{0}*N*. Observations were simulated by assigning each individual an annual sighting number from a uniform (0,1) distribution. If an individual's sighting number was lower than a fixed sighting probability (

_{0}*P*= 0.5, 0.7, 0.9, reflecting the range estimated from field data; see Table 1), it was considered observed for that year. By performing simulations with different sighting probabilities, we were able to imitate the effects of varying sampling intensity. Unobserved individuals could still reproduce, survive, and die. Simulations were run for a variable number of years (

*Y*= 5, 10, 20). All combinations of parameter values used are shown in Table 2. Simulations were repeated 200 times for each set of parameters.

Population Size N | Population Growth λ | |||||
---|---|---|---|---|---|---|

Observation | Jolly–Seber | Pradel | Observation | Jolly–Seber | Pradel | Sighting p Pradel |

538 | – | 1.128 | – | – | – | |

607 | 901 | 903 | 0.990 | 0.941 | 0.939 | 0.672 |

601 | 848 | 848 | 0.799 | 1.094 | 1.094 | 0.709 |

480 | 928 | 928 | 2.323 | 1.367 | 1.367 | 0.517 |

1,115 | 1,268 | 1,269 | 1.022 | 1.261 | 1.261 | 0.879 |

1,139 | 1,599 | 1,600 | 0.786 | – | 0.613 | 0.712 |

895 | – | 986 | – | – | – | 0.913 |

Estimates | Bias | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Parameters | ST | DE | JS | DE | JS | |||||||||

Sim. | N_{0} | N_{x} | p | Y | μ | σ^{2} | μ | σ^{2} | μ | σ^{2} | μ | σ^{2} | μ | σ^{2} |

^{}*Values significant at *P*< 0.01 in Tukey's HSD test.
| ||||||||||||||

A | 100 | 50 | 0.70 | 5 | 0.00201 | 0.00175 | 0.00330 | 0.0132 | 0.00796 | 0.0132 | 0.641 | 6.55* | 2.96 | 4.54* |

B | 100 | 50 | 0.70 | 10 | −0.000550 | 0.00192 | −0.000363 | 0.0115 | −0.000409 | 0.00596 | −0.339 | 5.01* | −0.0255 | 2.11* |

C | 100 | 50 | 0.70 | 20 | −0.00146 | 0.00188 | −0.00104 | 0.0114 | −0.00153 | 0.00514 | −0.0289 | 5.05* | 0.0482 | 1.73* |

D | 50 | 0 | 0.70 | 10 | −0.00110 | 0.00384 | −0.000829 | 0.0238 | 0.000507 | 0.0120 | −0.248 | 5.20* | −1.46 | 2.12* |

E | 200 | 150 | 0.70 | 10 | −0.000791 | 0.000957 | −0.000790 | 0.00568 | −0.000719 | 0.00310 | −0.00185 | 4.93* | 0.0908 | 2.24* |

F | 50 | 25 | 0.70 | 10 | −0.00175 | 0.00389 | −0.00130 | 0.0234 | 0.00130 | 0.0122 | −0.257 | 5.00* | −1.74 | 2.14* |

G | 200 | 100 | 0.70 | 10 | −0.000430 | 0.000956 | −0.000384 | 0.00579 | −0.000122 | 0.00320 | −0.106 | 5.06* | −0.716 | 2.34* |

H | 100 | 50 | 0.50 | 10 | −0.000122 | 0.00178 | 0.000718 | 0.0238 | 0.00138 | 0.0202 | −6.87 | 12.4* | −12.3 | 10.3* |

I | 100 | 50 | 0.90 | 10 | 1.45E-03 | 1.91E-03 | 1.43E-03 | 4.78E-03 | 7.21E-04 | 2.74E-03 | −0.0144 | 1.51* | −0.504 | 0.463* |

Our procedure simulated abundances in each year, providing us with a standard (henceforth referred to as simulated-true) against which to compare estimation by direct enumeration versus JS. We used these three types of simulated abundance data (simulated-true, direct enumeration and JS) to estimate μ and σ^{2} for use in the SEG. We then used each of the three SEG models in two different PVAs—direct simulation (using 1,000 replicates and parameters given in Table 2), and diffusion approximation (Dennis *et al*. 1991).

We calculated relative bias in μ and σ^{2}: the difference between direct enumeration or JS estimates and the simulated-true values, divided by the simulated-true values. We report bias in risk estimates as the simple difference between direct enumeration or JS and simulated-true risk, because extinction risk is a relative quantity and it is difficult to interpret statements about relative changes in a relative quantity. All simulations and analyses used R (R Development Core Team 2010).

### Results

#### Pine lily data

Table 1 shows the results of annual lily censuses (direct enumeration). JS and Pradel models provided alternative estimates of abundance, population trends, μ, and σ^{2}. Direct enumeration yielded estimates (95% CI) of population growth with μ = 0.0727 (−0.295, 0.441) and σ^{2} = 0.158 (0.0656, 0.766). In contrast, JS estimates were μ = 0.143 (−0.085, 0.371) and σ^{2} = 0.0271 (0.00972, 0.224). Using the diffusion approximation, extinction risk at 20 years dropped from 46.5% to 0.06% by using JS instead of direct enumeration (Figure 2a, c). Estimating risk with direct simulation gives risks of 35.7% and 0.01% using direct enumeration and JS, respectively.

#### Simulated data

Estimates of the average population trend—SEG parameter (μ)—based on direct enumeration, JS, and simulated-true population sizes, were not significantly different from one another regardless of simulated population sizes, length of sampling, or percent of the population observed (Table 2, Figure 3). In contrast, all simulations produced variance estimates (σ^{2}) that were significantly different from one another (Table 2). In every case, σ^{2} was smallest for simulated-true data, intermediate for JS, and largest for direct enumeration. We focused on a snapshot of 20 years after initiation to study the effects of μ and σ^{2} on risk. In nearly every simulation, the diffusion approximation resulted in higher risk estimates than direct simulation (Figures 3 and 4).

Increasing the sampling time decreased bias in JS and direct enumeration estimates of σ^{2} for all sample lengths (Table 2). Extended sampling time did not reduce bias in extinction risk based on the diffusion approximation for any of the three abundance data sources (Figure 4a). However, increasing the sampling time did reduce bias in direct simulation of risk based on direct enumeration and JS estimates; these reductions mirrored bias in σ^{2}, dropping from 23.3% to 12.3% and from 27.5% to 4.8%, respectively (Figure 4a).

Increasing the total population size did not affect σ^{2}, whether based on direct enumerations or JS. The effect of *N _{0}* on extinction risk depended on distance to the quasi-extinction threshold. When the threshold was

*N*– 50, bias in risk increased with population size using both abundance estimation methods (Figure 4b). However, when the threshold was

_{0}*N*bias was not affected by increasing

_{0}/2,*N*regardless of whether the diffusion approximation or direct simulation was used, or how abundance was estimated (Figure 4c). Increasing sighting probability decreased bias in σ

_{0},^{2}in both diffusion approximation and direct simulation of the SEG, unless true values of abundance were known (Figure 4d).

### Discussion

Our study of *L. catesbaei* produced four very different estimates (Figure 2) of extinction risk depending on the data used (direct enumeration vs. JS), and risk calculation (diffusion approximation vs. direct simulation). Using both types of risk calculation, JS gave lower estimates of extinction risk with confidence intervals that were narrower over short time frames, corresponding to lower variance among annual growth rates. Direct simulations with JS abundance estimates produced the lowest bias; based on these simulations (also see Kendall 2009), we conclude that the population of *L. catesbaei* at Brooker Creek Preserve has little danger of extinction within the next 20 years.

Why does mark-recapture lower our risk estimates? Stochasticity associated with sampling increases estimates of variance in the population trend (σ^{2} in the SEG), because SEG treats all stochasticity as environmental stochasticity. Although our application of JS to lily data resulted in higher μ, this does not appear to be a general feature of JS based on simulations. In fact, our simulations show that sampling stochasticity results in overestimation of risk, not through underestimating population size or μ, but through overestimating σ^{2} (Table 2, Figure 3). The JS model accounts for some of this sampling stochasticity when creating annual abundance estimates. Consequently, the SEG is parameterized with smaller year-to-year variance in population growth; all else being equal, this reduces the risk estimate (Morris & Doak 2002).

Wide confidence intervals are a common problem in PVA (Ellner *et al*. 2002; Fieberg & Ellner 2000; Holmes 2004; Ludwig 1999; Reed *et al*. 2002). In addition to other statistical methods for count data, (e.g., Holmes 2001; Holmes & Fagan 2002; Holmes *et al*. 2007; Buonaccorsi & Staudenmayer 2009), our results suggest that mark-recapture analyses can yield more precise estimates of extinction risk using count data. The method of estimating abundance with mark-recapture—JS or the Pradel method (Pradel 1996)—seems to have little effect on annual estimates of population growth rate (Table 1). We used JS because it is far easier to use in iterative simulations than the computationally intense Pradel model.

Our simulations show that abundance estimates resulting from JS yield parameterizations of the SEG model that are much closer to the true values than are the estimates from direct enumeration. The primary difference between JS and direct enumeration estimates of abundance is that direct enumeration is an index count, which will always underestimate true abundance with anything less than 100% sighting probability. JS is expected to eliminate some of the bias inherent in such index counts. While both methods yield accurate estimates of the average population trend, they overestimate its variance (Figure 3b). However, the variance estimates from JS were always closer to those of simulated-true abundances than estimates from DE, a fact not immediately deducible from consistent underestimation of population size expected of direct enumeration (Table 2). Where possible, it is most likely better to use JS than direct enumeration (Figure 3b).

What approaches are effective for reducing bias in estimates of σ^{2} and corresponding bias in risk? Increasing the length of sampling from 5 to 10 years decreased bias in σ^{2} (Table 2), which was mirrored in reduced extinction risk bias using direct simulation (Figure 4a). However, this did not decrease bias in extinction risk using the diffusion approximation. We suspect this is a result of nonlinear dynamics of the relationship between mean, variance, and extinction risk using the diffusion approximation.

Another route to improved predictions is to increase the total size of the population sampled, for example, by expanding the total geographical area sampled. However, in our simulations, increasing total population size had no effect on bias in variance; increasing sample size without increasing proportion of the population sampled seems unlikely to decrease bias in variance. We can intuitively confirm this by considering that the mean and variance of population growth rate are a stochastic realization of mortality, birth, and observation. Increasing the total number of individuals at the start of a simulation should not have any consistent effect on changes in population size from year to year. Therefore, the variability in σ^{2} in our simulations is simply the result of different stochastic realizations of the same underlying process.

How does sample size affect bias in risk estimates? To answer this, it helps to consider that risk estimates depend on both starting size *N _{0}* and the quasi-extinction threshold

*N*. We treated

_{x}*N*two ways: as a fixed amount smaller than

_{x}*N*, (Simulations B, D, E) and as a fixed proportion of

_{0}*N*(Simulations B, F, G) (Table 2). When

_{0}*N*is

_{x}*N*–50, increasing

_{0}*N*increases bias (Figure 4b). In contrast, when

_{0}*N*is

_{x}*N*/2, initial population size has no effect on risk bias (Figure 4c). All else equal, a large population is more likely to lose 50 individuals than a small population. However, populations of all sizes have similar chances of losing half their individuals. Accordingly, the extinction risk bias in proportional loss simulations is unaffected by starting population size (Figure 4c).

_{0}We considered one final approach researchers might take to reduce bias in risk estimates: increasing sampling intensity to increase sighting probability. This remedy is attractive because it keeps research within feasible time frames, and may work when increasing the area sampled is not an option. Increasing sampling intensity reduced bias in risk across the board (Figure 4d). This is the most consistent method we considered for decreasing bias in both σ^{2} and extinction risk, calculated with either of the two measures of abundance. However, there are cases where this is problematic, as with cryptic species like the pine lily.

Reducing observation error—by using mark-recapture, increasing the sampling time, or increasing the proportion of the population sampled—reduced bias in parameterizing the SEG and subsequently improved estimates of extinction risk in our simulations. This suggests that a high priority should be placed on accurately parameterizing the SEG with values of abundance less likely to bias PVA. As a result, we recommend use of mark-recapture, as well as altered sampling regimes. Furthermore, given identical data on abundance (whether from direct enumeration or JS), direct simulation always produced less biased estimates of extinction risk. Direct simulation and the diffusion approximation are fundamentally different ways of estimating extinction risk (Kendall 2009). Direct simulations of extinction risk showed improvement over the diffusion approximation in all cases, whether using direct enumerations or JS.

What does this imply for management of at-risk populations? First, we corroborate Kendall's (2009) findings, and suggest that extinction risk be estimated using stochastic simulations, rather than the diffusion approximation. Second, our results point to the importance of accuracy of population abundance estimates, even at the expense of other avenues for increased effort. Using mark-recapture to improve our abundance estimates reduced the bias in extinction risk in every simulation (Figure 3). Moreover, in our simulations, increasing the intensity of sampling a small population for a short time yielded more accurate results than sampling a larger total population, or for a longer period of time (Figure 4). Thus, management efforts might be better spent by intensively sampling a smaller population (increasing the sighting probability) rather than engaging in a more geographically dispersed effort. The improved extinction risk estimates that result from this effort may lead to more informed decision making.

### Acknowledgments

We thank B. E. Kendall, E. D. McCoy, S. Harper, and S. Bell for discussion, two anonymous reviewers for suggestions, and NSF for its generous support through grant DEB-0614468 to G. A. Fox.

### References

- 2001). Comparison of enumeration and Jolly–Seber estimation of population size in the common vole
*Microtus arvalis*. Acta Theriol., 46, 279-285. , , , , & ( - 2009). Statistical methods to correct for observation error in a density-independent population model. Ecol. Monogr., 79, 299-324. & (
- 1991). Estimation of growth and extinction parameters for endangered species. Ecol. Monogr., 61, 115-143. , & (
- 2002). Precision of population viability analysis. Conserv. Biol., 16, 258-261. , , & (
- 2000). When is it meaningful to estimate an extinction probability? Ecology, 81, 2040-2047. & (
- 2001. Estimating risks in declining populations with poor data. Proc. Natl. Acad. Sci. USA, 98, 5072-5077.
- 2004. Beyond theory to application and evaluation: diffusion approximations for population viability analysis. Ecol. Appl., 14, 1272-1293.
- 2002). Validating population viability analysis for corrupted data sets. Ecology, 83, 2379-2386. & (
- 2007). A statistical approach to quasi-extinction forecasting. Ecol. Lett., 10, 1182-1198. , , & (
- 2000). Restoration of Florida pine savanna: flowering response of
*Lilium catesbaei*to fire and roller-chopping. Nat. Area. J., 20, 12-23. & ( - 1965). Explicit estimates from capture-recapture data with both death and immigration-stochastic model. Biometrika, 52, 225-247. (
- 2009). The diffusion approximation overestimates the extinction risk for count-based PVA. Conserv. Lett., 2, 216-225. (
- 2003). Estimation of population growth and extinction parameters from noisy data. Ecol. Appl., 13, 806-813. (
- 1999). Is it meaningful to estimate a probability of extinction? Ecology, 80, 298-310. (
- 2002). Quantitative conservation biology: theory and practice of population viability analysis. Sinauer Associates, Sunderland, MA. , & (
- 1990). Ecosystems of Florida. University Press of Florida, Orlando. & (
- 1996). Utilization of capture-mark-recapture for the study of recruitment and population growth rate. Biometrics, 52, 703-709. (
- R Development Core Team, (2010). R: A language and environment for statistical programming, R Foundation for Statistical Computing. Available from: http://R-project.org. Accessed April 18, 2011.
*et al*. (2002). Emerging issues in population viability analysis. Conserv. Biol., 16, 7-19. , , - 2001). The Jolly–Seber model: more than just abundance. J. Agr. Biol. Environ. St., 6, 195-205. (
- 1999). Estimating animal abundance: review III. Stat. Sci., 14, 427-456. & (
- 1965). A note on the multiple-recapture census. Biometrika, 52, 249-259. (
- 2011). Inferring microhabitat preferences of
*Lilium catesbaei*(Liliaceae). Am. J. Bot., 98, 819-828 , , & ( - 2004). Estimating population trend and process variation for PVA in the presence of sampling error. Ecology, 85, 923-929. , & (
- 1999). Program MARK: survival estimation from populations of marked animals. Bird Study, 46, 120-138. & (