Site occupancy models in the analysis of environmental DNA presence/absence surveys: a case study of an emerging amphibian pathogen

Authors


Correspondence author. E-mail: benedikt.schmidt@ieu.uzh.ch

Summary

  1. The use of environmental DNA (eDNA) to detect species in aquatic environments such as ponds and streams is a powerful new technique with many benefits. However, species detection in eDNA-based surveys is likely to be imperfect, which can lead to underestimation of the distribution of a species.
  2. Site occupancy models account for imperfect detection and can be used to estimate the proportion of sites where a species occurs from presence/absence survey data, making them ideal for the analysis of eDNA-based surveys. Imperfect detection can result from failure to detect the species during field work (e.g. by water samples) or during laboratory analysis (e.g. by PCR).
  3. To demonstrate the utility of site occupancy models for eDNA surveys, we reanalysed a data set estimating the occurrence of the amphibian chytrid fungus Batrachochytrium dendrobatidis using eDNA. Our reanalysis showed that the previous estimation of species occurrence was low by 5–10%. Detection probability was best explained by an index of the number of hosts (frogs) in ponds.
  4. Per-visit availability probability in water samples was estimated at 0·45 (95% CRI 0·32, 0·58) and per-PCR detection probability at 0·85 (95% CRI 0·74, 0·94), and six water samples from a pond were necessary for a cumulative detection probability >95%. A simulation study showed that when using site occupancy analysis, researchers need many fewer samples to reliably estimate presence and absence of species than without use of site occupancy modelling.
  5. Our analyses demonstrate the benefits of site occupancy models as a simple and powerful tool to estimate detection and site occupancy (species prevalence) probabilities despite imperfect detection. As species detection from eDNA becomes more common, adoption of appropriate statistical methods, such as site occupancy models, will become crucial to ensure that reliable inferences are made from eDNA-based surveys.

Introduction

Population surveys have one undesirable yet unavoidable feature: it is unlikely that all individuals, populations or species are ever detected (Yoccoz, Nichols & Boulinier 2001; Pollock et al. 2002; Kéry & Schmidt 2008). This imperfect detection can bias analyses of survey data and potentially lead to poor management decisions (Yoccoz, Nichols & Boulinier 2001). It is therefore important to account for imperfect detection in the analysis of survey data whenever possible (Yoccoz, Nichols & Boulinier 2001; Pollock et al. 2002; Kéry & Schmidt 2008). Conceptually, the effects of imperfect detection can be described with a simple equation:

display math

where E(C) is the expected value of a count of individuals, populations or species, N is the number of individuals, populations or species actually present and p is the detection probability (Yoccoz, Nichols & Boulinier 2001; Pollock et al. 2002; Kéry & Schmidt 2008). Estimates of p can be used in the analysis of survey data to adjust counts for imperfect detection to obtain a more accurate estimate of N (Pollock et al. 2002; MacKenzie et al. 2006; Royle & Dorazio 2008). Apart from leading to biased assessments of N, not accounting for the probability of detection can result in over- or underestimation of the strength of influence of habitat characteristics on the occurrence of animals or plants (Mazerolle, Desrochers & Rochefort 2005).

MacKenzie et al. (2002) developed a species distribution model that can be used for presence/absence data (or rather detection/nondetection data, Royle & Kéry 2007) to estimate the proportion of sites where a species occurs, even when detection is imperfect. The key data requirement in these analyses is that there are multiple visits to a number of sites (e.g. MacKenzie & Royle 2005; Guillera-Arroita, Ridout & Morgan 2010) during a period when the true occurrence state (presence or absence) of a site is unlikely to change. The pattern of detection and nondetection at the sites then contains the information about detection probability, which allows one to estimate the true proportion of occupied sites (MacKenzie et al. 2002).

Many species are cryptic and hard to detect, so ecologists and conservation biologists have recently begun to survey the distributions of aquatic species using environmental DNA (eDNA; Brinkman et al. 2003; Walker et al. 2007; Ficetola et al. 2008; Jerde et al. 2011; Goldberg et al. 2011; Thomsen et al. 2012). This approach is based on detecting DNA of a target species in water samples taken from suspected habitats. The potentially high sensitivity and specificity of this method coupled with the often short period of DNA persistence in water (~2 weeks; Dejean et al. 2011) means species surveys using eDNA can provide a powerful method to test whether a species using an aquatic environment is present. Despite these advantages, most studies using eDNA still report imperfect detection (Ficetola et al. 2008; Hyman & Collins 2012; Thomsen et al. 2012). For example, Dejean et al. (2012) reported many ponds where only one or two of three water samples and sometimes only one of nine PCR samples revealed the presence of the target species.

Yoccoz (2012) argued that there is a need for new statistical models for the analysis of eDNA data because there can be various sources of error and uncertainty (e.g. sequencing errors, species identification, imperfect detection). We argue that site occupancy models have been developed already and can cope with multiple levels of uncertainty (MacKenzie et al. 2002; Royle & Dorazio 2008; Mordecai et al. 2011) and should be used to analyse eDNA survey data for two reasons. First, most studies using eDNA estimated detectability or sensitivity of the method using ad hoc comparisons with traditional surveying techniques. Although valuable, these comparative methods ignore the fact that both methods could fail to detect the species of interest (e.g. Hyman & Collins 2012; Thomsen et al. 2012). Therefore, the assessment of detection rates is too optimistic, similar to the naïve-low estimator described by Wintle et al. (2004). Occupancy models address this shortcoming by estimating detection probabilities and incorporating imperfect detection into species distribution estimates. In addition, most previous surveys based on eDNA from aquatic species have taken more than a single water sample per site and sampled multiple sites, resulting in data sets that are well suited for analysis by occupancy analyses. Secondly, previous studies assumed that species detection would be perfect, or almost perfect, if a sufficient number of water samples are taken per pond (Hyman & Collins 2012; Thomsen et al. 2012). Occupancy models provide an estimate of detection probability that could be used to determine the number of visits needed to be confident that a species is absent from a site (Kéry & Schmidt 2008).

In this article, we re-analyse a published data set (Hyman & Collins 2012) to show how site occupancy modelling can be used to analyse detection/nondetection data from surveys based on eDNA. We determine the best predictors of pathogen detection by water filtration, the number of water samples necessary to detect the pathogen with 95% confidence using our estimates of detection probability and then compare the new results with the published analysis that did not explicitly take imperfect detection into account. We expected that detection probability would be substantially smaller than one and that occupancy estimates would be higher than in the previous analysis. In addition, we conducted a simulation study to determine how many water samples were necessary to estimate species occupancy reliably.

Materials and Methods

Data collection

In 2010, we sampled 20 boreal chorus frog (Pseudacris maculata) breeding ponds for the presence of eDNA from the fungal pathogen Batrachochytrium dendrobatidis (Bd). Ponds were located in Coconino National Forest, Arizona, USA. For eDNA detection, we filtered approximately 600 mL of pond water from each pond at four different time points: (T1) when amphibian host breeding was initiated (March); (T2) 1–2 weeks postbreeding initiation (March–April); (T3) 3–4 weeks postbreeding initiation, when tadpoles were present (April–May); and (T4) 10 weeks postbreeding initiation when metamorphosed froglets began to emerge (June–July). We collected one filter from each pond at each time point, unless ponds had dried, in which case no filters were collected. Water was filtered and extracted following Kirshtein et al. (2007). The presence of Bd DNA in filter extracts was determined by qPCR analysis using internal positive controls (IPC; Hyatt et al. 2007) and bovine serum albumin (BSA; Garland et al. 2010) to control for and reduce PCR inhibition. Each sample was run in duplicate on a single qPCR plate. Each duplicate well contained BSA, but only one of each duplicate well contained IPC (see Hyman & Collins 2012).

Additionally, the occurrence of Bd was directly determined from skin swabs collected from approximately 30 breeding adult boreal chorus frogs from each pond during each of two time periods, T1 and T2 (as described above), for a total of approximately 60 animals sampled from each pond (details in Hyman & Collins 2012). DNA was extracted from swabs using Prepman Ultra and amplified by qPCR as described in Hyman & Collins (2012). Observed Bd prevalence (i.e. not adjusted for imperfect detection) at each pond was estimated as the total number of frogs testing Bd positive at each site divided by total number of frogs sampled from that site. Bd load within each pond at each time period (T1 and T2) was calculated as the mean load of all Bd-positive animals within the pond at the specified time period. We obtained a catch-per-unit-effort-type of index of frog density at each pond by calculating the animal capture rate at each pond, where:

display math

These quantities were used as covariates for the occupancy modelling (see below). All times were calculated from capture rates of a single person doing the sampling (OJH) to control for any biases in experience.

Data analysis

We fitted two variants of a site occupancy model to our data. The first model was the classical, two-level model of MacKenzie et al. (2002) and Tyre et al. (2003), which we fitted to an aggregated version of our data (see below) using both the program presence 3.1 (Hines 2006) and also the Bayesian modelling software winbugs (Lunn et al. 2000). In this model, we only distinguished between occupied and unoccupied sites and a single observation process; thus, the stochasticity due to the repeated water sampling and the error in PCR analysis was not separately modelled (nor were the parameters governing these processes). Hence, in a second analysis, we used winbugs to fit the three-level occupancy model of Nichols et al. (2008) and Mordecai et al. (2011), which distinguishes all three stochastic levels of the process underlying our observed data: (i) the occupancy process leading to some ponds being occupied and others being unoccupied; (ii) the water sampling process resulting in samples that contain the DNA of the respective species and others that do not; and (iii) the PCR sampling process, which results in a detection of a species or not given that DNA was available for detection in the water sample (see McClintock et al. 2010 for discussion of such hierarchical sampling schemes for disease surveillance).

Our data are yijk, the binary indicators of detection (1) or nondetection (0) of the species at site i (i = 1,…,20), in water sample j (j = 1,…, 4) and PCR analysis k (= 1,2 for the two duplicate wells). For the traditional two-level occupancy model, we aggregated the results from the two PCR analyses and thus analysed yij, that is, the binary indicators for whether the pathogen species was or was not detected in either of the two PCR analyses conducted for site i and water sample j. Ignoring possible covariates for ease of presentation, the traditional, two-level site occupancy model can then be written concisely in just two lines:

display math
display math

The first equation defines the latent state of occurrence of the species in pond i as a Bernoulli trial with success parameter ψi (occupancy probability), which corresponds to the frequency of Bd-positive sites in the ‘population’ of sites from which the study sites were sampled. The second equation defines the false-negative measurement error where zi is the latent occurrence state (either 0 or 1) and pij detection probability. To make the model identifiable, constraints need to be imposed on the detection model, typically in the form of a linear model specified for logit (pij) and similarly also for the logit transform of occupancy probability (ψi).

The three-level occupancy model by Nichols et al. (2008) and Mordecai et al. (2011), again ignoring covariates simply for ease of presentation, consists of a sequence of three coupled Bernoulli trials as a description of the full three-dimensional array of data yijk.

display math
display math
display math

The first random variable, zi, again describes the presence (zi = 1) or absence (zi = 0) of the species at pond i as a function of the occupancy probability ψi. The second random variable, aij, describes the presence (aij = 1) or absence (aij = 0) of DNA of the species in water sample j taken from site i as a function of the occurrence state of the species at site i, zi, and the availability probability θij. Note that when zi = 0 (i.e. the species is absent from a site), aij will be sampled from a Bernoulli distribution with a success probability equal to zero and will be always zero. This component of the model describes the probability that Bd eDNA is available for detection in water sample j from site i. The third random variable is the observed data, yijk, governed by the presence of the species in water sample j from site i, aij and the detection probability given Bd DNA is in the water sample pijk. An implicit assumption of this model component is that there are no false positives. Thus, the model decomposes the probability of detecting the pathogen at ponds where it exists into two components: an availability (or sampling) component (i.e. the probability that the pathogen DNA is captured in a water sample j) and a (strict) detection component (i.e. the probability that the DNA is detected in a PCR assay of a water sample that contained the pathogen's DNA k). In the two-level model, the detection probability that is being estimated is the product of availability and detection probability. Covariates can be introduced in the usual generalized linear model manner at any of the three levels of the hierarchy, typically using a logit link function (MacKenzie et al. 2002; Nichols et al. 2008; Mordecai et al. 2011).

In presence, we fitted the classic two-level site occupancy model (MacKenzie et al. 2002; Tyre et al. 2003) to the data and ranked candidate models according to Akaike Information Criterion (AIC; Burnham & Anderson 2002). Based on the available data (Hyman & Collins 2012), we defined a small set of candidate models (Table 1; model notation follows MacKenzie et al. 2002). Because the analysis by Hyman & Collins (2012) showed that Bd occurred in most studied ponds, we included no covariates for occupancy (i.e. fitted a model psi(.) with constant occupancy probability). Models for detection probability (θ) included one of the covariates ‘frog density index’, ‘load T1’, ‘load T2’, ‘Bd prevalence’ (hereafter, we denote these models by p(‘covariate’), where ‘covariate’ stands for one or several of the covariates in the linear predictor of detection). We also fitted a model where detection probability was constant across sites and visits (p(.)). Finally, because Hyman & Collins (2012) found evidence for seasonal variation in rates of Bd detection from water samples, we also fitted a model with visit-specific detection probabilities (p(t)).

Table 1. Model selection results for two classic, two-level site occupancy models. ΔAIC is the difference between the model with the lowest AIC value and the focal model, Akaike weight is the probability that a model is the best in the set of candidate models. K is the number of parameters in the model
ModelΔAICAkaike weight K −2log-likelihood
psi(.)p(frog density index)00·456393·76
psi(.)p(t)2·040·164591·80
psi(.)p(load T2)2·530·129396·29
psi(.)p(Bd prevalence)3·030·100396·79
psi(.)p(load T1)3·440·082397·20
psi(.)p(.)3·780·069299·54

After ranking the different candidate models in presence based on AIC, we used the hierarchical parameterization of the three-level site occupancy model of Mordecai et al. (2011) to estimate model parameters in a Bayesian way and obtain their 95% credible intervals (95% CRI); we note that a frequentist analysis of the model can be conducted in program presence. We used finite sample inference as described by Royle & Kéry (2007) to estimate the number of ponds in which Bd occurred in our studied sample. For this modelling, we used programs r and winbugs (Lunn et al. 2000; Sturtz, Ligges & Gelman 2005; R Development Core Team 2009). winbugs code to fit the model is available in Mordecai et al. (2011). For all model parameters, we used vague uniform priors. For occupancy, the prior was ψ ~ dunif(0,1), whereas the priors were β0 ~ dunif(−15,15) and β1 ~ dunif(−15,15) for the coefficients β0 (intercept) and β1 (slope) of the logistic regression describing the relationship between availability probabilities (θi) and the covariates (we observed that the MCMC converged much more rapidly with uniform than with comparative vague normal priors). For the model with time-varying detection probabilities, we used uniform priors: θj ~ dunif(0,1). For detection at the level of the PCR, the prior was p ~ dunif(0,1). This probability was held constant across sites, water samples and replicate PCRs. For each model, we ran three Markov chains with 10 000 iterations each, discarded the first 1000 iterations as burn-in and thinned the remainder by one in five. Convergence was assessed using the Brooks–Gelman–Rubin statistic (Brooks & Gelman 1998).

We used the mean availability probability at the water sample level (inline image) to calculate the cumulative probability of detecting Bd after 1, 2, …, n water samples (p*) based on the equation of McArdle (1990)

display math

This equation assumes that the species is present at the site (see Wintle et al. 2012 for the more general case that does not condition on species presence). By doing all the computations in winbugs as part of model fitting, the estimate of p* directly accounts for uncertainty in all parameter estimates.

Simulation experiment

We conducted a simulation experiment to gauge the quality of the estimate of the probability of occupancy (ψ) when fewer than four water samples were taken at each site. For this, we subsampled our dataset such that we randomly, and with replacement, chose 4, three or only two water samples per site. Then, we fitted the three-level occupancy model psi(.)θ(t)p(.) and saved the summary of the posterior samples of ψ. We repeated this 1000 times.

Results

Based on the two-level model, the number of frogs captured per minute (our index of frog density) as a covariate for detection probability was best supported by the data (Table 1). Other models had considerably less support, including the model that contained time-specific trends in detection probability (psi(.)p(t)), which was emphasized by Hyman & Collins (2012). However, regardless of the model and its rank, based on the three-level model, the estimated proportions [range: 0·88 (95% CRI: 0·69, 0·99)–0·92 (95% CRI: 0·75, 1·00)] and the estimated number of ponds where Bd occurred [18 (95% CRI: 17, 20) or 19 (95% CRI: 17, 20)] were all very similar, with most models suggesting that Bd was present in ponds (Table 2). Hyman & Collins (2012) reported that using eDNA methodology, they detected Bd in 17 ponds.

Table 2. Parameter estimates from the three-level occupancy model of different candidate models shown in Table 1. Table entries are posterior means and 95% CRI
ModelOccupancy probability ψEstimated number of ponds where Bd occursAvailability probability in water sample θaDetection probability in PCR p
  1. a

    Table entries are either probabilities (models psi(.)θ(t)p(,) and psi(.)θ(.)p(.)) or estimates of the coefficients of the relationship between the covariate and detection probability of the form logit(θ) = β β1* covariate.

psi(.)θ(frog density index)p(.)0·92 (0·75, 1·00)19·3 (17·0, 20·0)

β0 = −1·96 (−3·75, −0·26)

β1 = 5·14 (0·57, 10·23)

0·85 (0·73, 0·94)
psi(.)θ(t)p(.)0·88 (0·69, 0·99)18·3 (17·0, 20·0)0·37 (0·17, 0·60), 0·61 (0·38, 0·82), 0·64 (0·41, 0·86), 0·25 (0·08, 0·49)0·85 (0·73, 0·94)
psi(.)θ(load T2)p(.)0·92 (0·75, 1·00)19·3 (17·0, 20·0)

β0 = −0·76 (−1·67, 0·05)

β1 = 1·80 (0·10, 4·61)

0·83 (0·70, 0·93)
psi(.)θ(Bd prevalence)p(.)0·92 (0·76, 1·00)19·2 (17·0, 20·0)

β0 = −0·72 (−1·67, 3·64)

β1 = 1·55 (−0·52, 3·64)

0·85 (0·74, 0·94)
psi(.)θ(load T1)p(.)0·91 (0·78, 1·00)19·1 (17·0, 20·0)

β0 = −0·39 (−1·01, 0·24)

β1 = 1·96 (−0·48, 6·25)

0·85 (0·73, 0·94)
psi(.)θ(.)p(.)0·90 (0·70, 1·00)18·8 (17·0, 20·0)0·461 (0·34, 0·59)0·85 (0·74, 0·94)

Based on the three-level model, the relationship between the frog density index and detection probability (Fig. 1) was positive and the 95% credible interval of the slope estimate did not include zero (Table 2). Mean availability probability (at the level of a water sample) under this model was estimated at 0·45 (95% CRI: 0·32, 0·58). Cumulative availability probability after 1, 2, …, 10 samples is shown in Fig. 2. In ponds where Bd is present, six water samples were necessary to obtain a cumulative availability probability >0·95. The estimate of detection probability at the level of the PCR was 0·85 (0·74, 0·94). Parameter estimates for the other models in our candidate set are shown in Table 2. The estimated occupancy probability was 0·92 (0·75, 1·00), and the estimated number of ponds with Bd was 19·3 (17·0, 20·0).

Figure 1.

Relationship between frog density index and availability probability based on the model psi(.)θ(frog density index)p(.). Symbols are means and 95% credible intervals. Availability probabilities were estimated for frog density indices observed at the 20 ponds.

Figure 2.

Relationship between the number of samples taken from a pond and cumulative availability probability (assuming that the species is present at the focal site). Symbols are means and 95% credible intervals. Cumulative availability probability is calculated based on the mean availability probability estimated under the psi(.)θ(frog density index)p(.) model [inline image = 0·45 (95% CRI: 0·32,0·58)]. The horizontal dotted line shows where the cumulative availability probability is 0·95.

The simulation experiment showed that the point estimates (posterior means) of Bd occupancy probability (ψ) became more spread out when going from 4 to 3 and then two water samples per site and that the estimation uncertainty (posterior standard deviation) increased (Fig. 3). Nevertheless, estimates would have been fairly reasonable with three or even only two water samples taken per site. This result suggests that a similar assessment could have been made with only three or two samples per site, resulting in reduced costs, or, alternatively, that additional ponds could have been sampled. Nevertheless, as we were almost certain to detect Bd DNA by PCR when taking two samples (i.e. 1 − (1 − inline image)2 = 0·98), the reliability of a study will increase with additional water samples rather than additional PCR.

Figure 3.

Results from the simulation experiment to gauge the quality of the estimates of Bd occupancy probability when two, three or four water samples are taken per site. Left: distribution of 1000 posterior means, right: distribution of 1000 posterior standard deviations. The vertical solid line shows the mean of the 1000 estimates, and the vertical dotted line shows the mean of the 1000 observed occupancy probabilities (i.e. ‘naïve’ occupancy that does not take imperfect detection into account).

Discussion

New statistical methods are necessary if we want to take full advantage of eDNA data (Yoccoz 2012). The analysis and results presented here show that eDNA data can easily be analysed in a statistically sound way using site occupancy models if multiple samples per pond are taken (MacKenzie et al. 2002; Tyre et al. 2003; Royle & Dorazio 2008). Our re-analysis of the data presented and analysed in Hyman & Collins (2012) showed that Bd occurred in more ponds than estimated by Hyman & Collins (2012) using methods that did not explicitly account for imperfect detection. Hyman & Collins (2012) used two methods to detect Bd in ponds: eDNA and swabs of individual frogs. With both methods, they detected Bd in 17 of 20 ponds. There were, however, only 16 ponds where they detected Bd with both methods. Based on the results of the site occupancy analysis, there is a high probability that Bd was present in at least 19 ponds, suggesting a small underestimation of pond occupancy by Hyman & Collins (2012). Per-sample detection probability using eDNA from water samples was low [0·45 (95% CRI: 0·32, 0·58)], but cumulative detection probabilities after four water samples were high (Fig. 2). This explains the small difference between the estimate of the number of ponds with Bd obtained by Hyman & Collins (2012) and our new results.

Imperfect detection of species by means of eDNA seems to be the rule rather than an exception. An analysis of the data presented in the electronic appendix of Thomsen et al. (2012) shows that per-visit detection probability was often considerably smaller than one (based on the two-level model, the lowest estimate [0·42 (95% CI: 0·27, 0·60)] was for the dragonfly Leucorrhinia pectoralis and the highest estimate [0·83 (95% CI: 0·64, 0·93)] for the newt Triturus cristatus). Thus, to reliably infer presence or absence of a species at a site and the proportion or number of sites where a species occurs, multiple samples per pond will be necessary (Fig. 2). Our simulation study (Fig. 3) suggested that when using site occupancy models, as few as two samples may be sufficient for reliable occupancy estimation. Thus, occupancy estimation offers great benefits to researchers. If there is a fixed amount of money for field and laboratory work, then instead of visiting every site four times, one may visit more sites three or two times. For example, instead of visiting 20 sites four times (as in Hyman & Collins 2012), one might visit 25 sites three times or as many as 40 sites only twice. Reducing the number of site visits (i.e. water samples per site) leads to wider credible intervals (Fig. 3) but the simulations of MacKenzie & Royle (2005) and Guillera-Arroita, Ridout & Morgan (2010) lead us to expect that adding more sites to a study would counteract this effect. We believe that for many ecological questions that may be asked using eDNA methodology, it may be better to sample more sites less intensively.

Thus, explicit modelling of detection probability allows one to estimate the number of occupied ponds based on a much smaller effort than when one has to sample them exhaustively. Clearly, the latter can be performed: there is always some level of effort at which any uncertainty about pond occupancy is essentially eliminated and the true state of occurrence of a species at each site becomes known perfectly. However, such perfect knowledge will be costly. In our case, six water samples per pond were required to achieve a virtually exhaustive sampling, but in other studies, with other species, other sampling protocols etc., the effort required for achieving an exhaustive sampling may be higher. Thus, one key benefit of occupancy modelling is that it can save (much) money. Even if the goal is exhaustive sampling, occupancy modelling enables formally testing the assumption of the exhaustiveness of a sample rather than having to make untested assumptions. Hence, we strongly recommend occupancy modelling for eDNA studies.

In addition, an important feature of occupancy models is the possibility of testing for the effects of site characteristics on occupancy or detection probabilities (MacKenzie et al. 2002). For eDNA, animal density may be an important site characteristic because it may determine the concentration of eDNA in water (Thomsen et al. 2012). For site occupancy models, this is an important finding because theory predicts that detectability of species should depend on their abundance (Royle & Nichols 2003; Tanadini & Schmidt 2011). Here, we found that the detection probability depended on the frog density index (Fig. 1). Although both host density and pathogen prevalence are likely to influence the concentration of Bd DNA in water, our result suggests that host density may be a better predictor than pathogen prevalence. This is in contrast to Goldberg et al. (2011) who found no relationship between densities of Rocky Mountain tailed frogs (Ascaphus montanus) and the probability of eDNA detection but in agreement with Thomsen et al. (2012) who described a positive relationship between focal species density and eDNA concentration.

Several previous studies have tested the sensitivity of eDNA methods to detect the DNA of target species in water samples (Ficetola et al. 2008; Dejean et al. 2011; Goldberg et al. 2011; Hyman & Collins 2012; Thomsen et al. 2012). One approach was to use two different and independent methods to verify species presence. For example, Hyman & Collins (2012) used swabs of frogs to detect Bd as their second method, whereas Thomsen et al. (2012) used visual encounter surveys to detect species. These comparative approaches are valid but ignore the risk that the target species will be missed by both methods. The risk of not detecting a species cannot be accurately quantified using the approaches described in Hyman & Collins (2012) and Thomsen et al. (2012), because these approaches are not based on an inferential statistical model. Site occupancy models address this weakness by providing a measure of uncertainty (e.g. a 95% credible interval).

In our case, per-sample detection probability was 0·43 (95% CRI 0·31, 0·56), so multiple samples per pond will be necessary to be 95% certain that Bd will be detected (Fig. 2; McArdle 1990). Generally speaking, if the sampling design includes multiple samples for every pond (or at least at a subset of the ponds; MacKenzie & Royle 2005), then we recommend that the data should be analysed using a site occupancy model to account for imperfect detection.

While site occupancy models are certainly very useful for the analysis of eDNA data, there are also some caveats. Four issues in particular should be discussed: small sample size, detection heterogeneity, the closure assumption and false positives. As in all statistical estimation based on maximum likelihood or other criteria, a small sample size can lead to bias or large standard errors also in occupancy models (MacKenzie et al. 2002; Guillera-Arroita, Ridout & Morgan 2010). We would generally recommend that at least 20 sites (as in this study) should be surveyed. General recommendations for the design of site occupancy studies and description of how to construct simulation studies can be found in MacKenzie & Royle (2005), Bailey et al. (2007), Guillera-Arroita, Ridout & Morgan (2010) and Kéry & Schaub (2012).

Heterogeneity in detection probability can lead to biased estimates of occupancy (Dorazio 2007). Tacitly, the classical site occupancy model used here assumes equal population size at all sites or that detection probability is independent of the population size at occupied sites (Royle & Nichols 2003). This is certainly not a realistic assumption in most cases. Thus, it may be worthwhile to account for heterogeneity in the detection process among sites in the analysis. If there are no environmental covariates that can be used to model spatial variation in detection probability, then models with random effects may be used (Royle 2006). Recently, Lachish et al. (2012) and Miller et al. (2012) provided methods to deal with heterogeneity in the context of disease ecology.

Occupancy models assume that the sites are ‘closed’, which means no changes in occupancy status during the survey period (MacKenzie et al. 2002; Rota et al. 2009). Rota et al. (2009) argued that the closure assumption was often violated and that certain types of ‘lack of closure’ (corresponding to what is called non-Markovian temporary emigration) lead to bias in site occupancy estimates. Because eDNA persists in water for a few days (Dejean et al. 2011), the closure assumption should easily be fulfilled assuming that the focal species occurs for the duration of the sampling period. For example, if one studies frogs that migrate to the breeding site (a pond) in spring, eDNA should be permanently present from shortly after the arrival of the first frogs until tadpoles complete metamorphosis. To be on the safe side, eDNA data should probably not be collected very early in the breeding season of a species because eDNA concentrations build-up slowly (Thomsen et al. 2012). In general, a sound knowledge of the natural history of the species under study will help to determine when samples for an occupancy study should be collected.

A species may not only be missed during a survey; false positives are also possible. Royle & Link (2006) showed that false positives can lead to strong bias in the occupancy estimator. If false positives are not accounted for, then occupancy probability is overestimated. Miller et al. (2012) modified site occupancy models to fit site occupancy models with false positive errors. The occurrence of false positives has not yet been fully assessed for eDNA (but see Yoccoz et al. (2012)), but given the strong biases they can create, laboratories should adhere to strict protocols to prevent and potentially account for sample contamination including the use of negative extraction and PCR controls, dedicated laboratory space, replication of sample amplification by independent laboratories and potentially avoiding the use of positive controls (Cooper & Poinar 2000).

eDNA data can be collected in such a way that the assumptions of the site occupancy model are met. For example, (i) sites must not change occupancy status (species either present or absent; the ‘closure’ assumption), while field work is conducted (see above) and (ii) detection at one site must be independent of detection at other sites (MacKenzie et al. 2002). The basic model of MacKenzie et al. (2002) has been extended to deal with many additional biological questions. For example, the multiseason site occupancy model that estimates site occupancy, colonization and extinction probabilities can be used to analyse spatiotemporal dynamics of species occurrence (MacKenzie et al. 2003). The flexibility and capacity to account for imperfect detection make site occupancy models a useful and robust modelling framework to overcome many of the computational challenges originating from eDNA survey data (Yoccoz 2012). Our results demonstrate the advantages of occupancy models over typical analyses of eDNA data, and we recommend their incorporation into the design and analysis of species surveys that rely on methods with imperfect detection, such as eDNA.

Acknowledgements

We thank Michael Schaub for help with winbugs programming and the reviewers and the editors for helpful comments on previous versions of the manuscript. Barbara Garcia, Jill Oertly and the United States Forest Service provided logistical support and field assistance for Bd surveys. Crystal Meins helped with PCR analyses. All work was approved by Arizona State University IACUC (09-092R) and funded by Heritage Grant I10018 to O.J.H. and J.P.C. from the Arizona Game and Fish Department.

Ancillary