A Novel Tool for Making Policy Recommendations Based on PVA: Helping Theory Become Practice

Mathematical models have long been used to aid in conservation decision‐making, however there are many objections to their use by conservation practitioners. Two common objections are that model outputs are too uncertain and are often not communicated effectively. Population viability analysis (PVA) often makes use of mathematical models and has become a mainstay in conservation management; however, it is not immune to these problems. Here we provide a simple method for using the output of PVA models to calculate the probability of management achieving “success” and “failure,” as well as the probability that implementing management will be no more effective than doing nothing at all. Using our method with 14 previously published PVAs, we show that expected probability of success varied from 0.14 to 0.98, failure from 0.054 to 0.6, and the probability management was not needed varied from 0.015 to 0.47. Calculating and reporting these probabilities provide conservation practitioners and policy‐makers with more intuitive and tangible ways of identifying potential risks and rewards to their actions.


Introduction
Conservation practitioners are tasked with making difficult decisions regarding the protection of threatened species. Mathematical models, such as those commonly used in population viability analyses (PVA), are often used to make plausible predictions about the effectiveness of management actions. While such models have become a common tool in decision-analysis, model results can be difficult to assess and understand by nonmodelers (e.g., research-implementation gap; Knight et al. 2008;Pe'er et al. 2013). Addison et al. (2013) found that two common objections to using models were that decision-makers lacked confidence in model outputs because of the uncertainty inherent in their construction, and that modelers did not communicate model outputs in a manner that made it clear how results informed trade-offs between costs and benefits (Borowski & Hare 2007;Addison et al. 2013). Here we provide a method to help bridge the gap between conservation practitioners and modelers that allows the output of PVA models, and the uncertainty associated with them, to be communicated and evaluated in a more intuitive manner.
Although increasingly complex modeling methods are becoming more widely used (e.g., Lacy et al. 2013), a very common form of PVA uses a population projection matrix consisting of vital rates to detail life histories of target species (e.g., age-specific growth, survival and fecundity rates). PVA users mimic possible management actions by perturbing vital rates in the model via a global (e.g., McCarthy et al. 1995;Wisdom et al. 2000;Cross & Beissinger 2001) or a local (one-at-a-time) sensitivity analysis, with a global approach considered superior (Naujokaitis-Lewis et al. 2008;Saltelli & Annoni 2010). The change in vital rate(s) that results in the largest desired change in one of the many measures of viability (e.g., probability of extinction/quasi-extinction, probability of decline, population size at a given time, minimum expected population size, growth rate [λ], mean time to extinction) can then be targeted by practitioners for action (Possingham et al. 1993;Wisdom et al. 2000;McCarthy & Thompson 2001;Grimm & Wissel 2004;Pe'er et al. 2013).
There are a variety of sources of uncertainty in models, and the need to incorporate these into matrix-based PVA models has long been recognized (Boyce 1992). Methods for doing so continue to advance (e.g., Kendall 1998;White et al. 2002;McGowan et al. 2011;Heard et al. 2013). No matter the mechanism, including stochastic elements in PVAs produces a distribution of the chosen measure of viability. Therefore, evaluating the potential impacts of management by perturbing one or more vital rates will produce a distribution of possible population projections. When conservation management is applied in real life, however, the only realized outcomes are the ones we observe.
The often-unstated assumption in making recommendations from PVAs is that implementing management that mathematically shifts the mean or entire distribution of a viability metric upward will necessarily achieve an observed increase in that metric. However, in many cases there will be a portion of the initial distribution of the metric (unmanaged) that overlaps with the final distribution of the metric (managed). By only considering the shift we create in the distribution of a chosen measure of viability via management, or shift in the mean of these distributions, we are ignoring the amount of overlap the final (after management) and initial (absent management) distributions have with one another. The degree to which these two distributions overlap defines two distinct probabilities. The first is the probability that doing nothing will result in the same outcome as enacting management (e.g., Robinson et al. 2013). The second is that implementing management will increase the metric, but will not push it to the target value; in which case management simply will not achieve the desired goal.
Explicitly defining and reporting these probabilities may enhance the usefulness of PVA by providing ways to calculate various risks associated with conservation actions (including no action). If these risks are made transparent to conservation practitioners and policy-makers, these individuals have a more complete tool-kit at their disposal for making difficult judgments on balancing the risks versus rewards of their decisions. To address this shortfall, we suggest a two-step approach where the initial step is to make the conservation goal explicit by setting a target value for the viability metric, thereby creating a minimum threshold one is willing to accept from management action given funding, time, and resources. The second step is calculating the probability that proposed management actions will in fact achieve this value in any one realization of the model.

Methods
We searched Web of Science R for published PVAs using the search term "population viability analysis" over the years 1995-2013. Searches were limited to journals containing "Conservation" and/or "Management" in their title. We retained the first 50 publications that reported a PVA for one or more species, returned in inverse order of publication year. These 50 publications ranged in publication date from 1999-2011. Of those 50, only 11 met our criteria for inclusion. This poor reporting and repeatability in PVA has long been a problem (Pe'er et al. 2013). We included articles that reported survival and fecundity estimates for each age/stage class, made a statement of population status or reported λ prior to management, gave the data by which one could calculate λ (i.e., was the population declining, stable, or increasing before management), used λ (or population growth) as a viability metric, reported a clear management goal, and made an unambiguous management recommendation based on the PVA performed. We chose λ as the viability measure on which to focus here; however this does not mean that this viability measure should be preferred over others among the many available viability measures (e.g., probability to extinction, mean time to extinction, minimum expected population size, etc.reviewed in Pe'er et al. 2013). Our approach should be applicable for almost all of the measures attained through repeated stochastic realizations.
We recreated PVA models for the 14 populations (12 different species) from the data described within these 11 publications (Table 1), recording the distribution of λ before and after the interventions recommended for management in each article. We used a linear regression to compare the mean λ reported in each publication to the mean λ of our distributions to verify that the models we developed gave similar results to the models presented in each publication (see Results). We recorded from each publication the original author's stated conservation goal in terms of the value of λ they desired based on the model they presented; i.e., if the model claimed a λ value of 0.95 was achievable by management, we used λ = 0.95 as the target. When a value greater than 1 was achievable by management, we used λ ࣙ 1 as the target. When no target λ value was given, for example, when an article suggested that management would stem the decline of a population, improve λ, or achieve a stable population, we assumed the target to be λ ࣙ 1. We recognize that often the goal of specific management actions is not always λ ࣙ 1, and we realize that using the target of λ ࣙ 1 will inflate the number of times that management does not achieve its goal when analyzed by our method. The goal of this decision was not to determine whether a specific  a Article either gave specific number for λ, or stated that the population was in decline. b Article did not give a specific target for λ so a λ = 1.00 was used as the target. * Four separate population models within the manuscript Perlut et al. (2008) were evaluated.
recommendation was "right" or "wrong," but simply to present how our method would be applied to PVA results. When authors suggested multiple management options could achieve a conservation goal, we chose the most optimistic for use in our evaluation. This choice produces the most optimistic probabilities of attaining success for each study. When authors suggested that multiple management strategies should be implemented simultaneously to achieve a stated goal, or when one management strategy was able to affect multiple vital rates in concert, we allowed those vital rates to be affected together. We did not determine if a stated management strategy was feasible.
We utilized our model to produce two distributions of λ for each of the 14 populations gleaned from the above publications. One of these distributions is produced absent of any management action, and the other distribution is produced by incorporating the suggested Schematic diagram of the output generated from PVA, which is composed of a range of possible population growth rates (λ). The distribution of growth rates is a product of incorporating environmental and demographic stochasticity, and thus reflects the range of growth rates deemed reasonable given our understanding of natural variation in the system. We evaluated the output of 14 published PVAs using two distributions of λ values: one calculated without management actions in effect (dashed line), and another calculated with management actions in effect (solid line). We designated target growth rates (vertical dotted line) for each PVA based on information provided in each publication. This example is for a target λ of 1. Based on these factors, we calculated three outcomes: (1) the probability that growth rate was increased from a declining value to at least an increasing value (2) the probability that the growth rate was above the target value before management was enacted (gray shading), and (3) the probability that the growth rate remained below the target after management was incorporated (black shading).
management. These distributions were the result of paired stochastic draws from the vital rates. That is, for a given model realization, a random matrix was drawn to determine λ for a nonmanaged population. Management was then simulated on that same random matrix to determine λ for a managed population. We then affixed the author-stated target value of λ (if less than 1) to each distribution (see above). This allowed us to quantify a number of meaningful metrics: (1) Success = the number of times out of 10,000 realizations in which intervention pushed an initial λ from below the target value to a managed value equal to, or larger than, the stated target (2) Not Needed = the number of times out of 10,000 realizations that λ for the original, unmanaged population overlapped with the λ values obtained for the managed population at values greater than the target λ (gray shading within Figure 1). (3) Failure = the number of times λ remained below the target value even after including the positive effects of a recommended management action (black shading within Figure 1). The frequency of each of these three outcomes is dictated by how far below the target growth rate the population sits premanagement, and the extent to which suggested management actions can move growth rates up to and past the stated target value (Figure 1). We represent the frequency of each of the three outcomes above within pie charts, where the shading matches the corresponding outcome. Success indicates the probability that a recommended management action achieved its stated goal. Not Needed indicates the frequency with which a population growth rate was already equal to, or exceeded, the target growth rate before the effects of management were included, Failure indicates the frequency of cases in which enacting management failed to move growth rates at least to the target value.
By combining the number of realizations that ended in outcomes Success and Failure above, we determined the number of realizations in which management was needed (i.e., λ was below the target value before management was applied). We then computed

Suc c ess
Suc c ess+F ailure to determine how often management is expected to achieve stated goals, given that it is needed. We also computed Success + Not Needed to capture the number of realizations that ended in the achievement of the target growth value, no matter how that value was attained.

Results
The regression analysis of the premanagement λ values presented for each article (the response variable in our regression) versus those given by our model produced a y-intercept of 0.087 and β = 0.91 (R 2 = 0.94). This suggests that our model slightly underestimated those λ at the low end of the range of tested values and slightly overestimated λ at the high end of the range. There was a similar pattern in the post-management values as well (y-intercept = 0.083; β = 0.92, R 2 = 0.99).
Our results reveal widely differing probabilities for each of the three possible outcomes (Table 2; Figure 2). The probability that any one realization resulted in λ reaching a predetermined target value (Success; white shading) ranged from 0.14 to 0.98. In only two of the 14 PVAs did at least half of the realizations fall into this category ( Figure 2). The probability that any one realization resulted in a growth rate that was at or above the target value of λ, without accounting for the positive effects of management action (Not Needed; gray shading), ranged from 0.015 to 0.47 (Figure 2). This is interpreted as how often the conservation goal was attained without any management action. Finally, the probability that any one realization resulted in λ failing to attain the target value even when the positive effects of management are included ranged from 0.054 to 0.60 (Failure; black  Table 2 The probability that one PVA realization out of 10,000 that resulted in one of three possible outcomes (see text, Figure 2). Also included are the probabilities that λ < target and thus management was needed, the probability that λ rose above the target when starting < target (management worked when needed), and the probability that λ > target regardless of management shading, Figure 2). This probability is interpreted as how often management was applied to a population but failed to achieve the conservation goal. We choose three example PVAs to illustrate how the results in Figure 2 can also be displayed as shifts in distributions (as in Figure 1). The three examples we choose represent ends of the spectrum of outcomes. Thus, we show the distributions that created a pie chart that was mostly black (Figure 3a), one that was mostly white (Figure 3b) and one that was mostly gray (Figure 3c). Displaying the results of a PVA in this way (e.g., life-stage simulation analysis; Wisdom et al. 2000) allows one to see the shift in the distribution that is possible, while our pie charts ( Figure 2) allow one to see the probability that the shift may result in reaching a conservation target.
By combining the results above, we calculated that the probability that management was needed (Success + Failure) ranged from 0.26 to 0.99. For all but one of the PVAs, management was required in more than half of the realizations (Table 2). Management was expected to work when needed (

Suc c ess
Suc c ess+F ailure ), with a probability that ranged from 0.25 to 0.99 (Table 2). Finally, the probability that λ was equal to or exceeded the target value regardless of whether management was applied (Success + Not Needed) ranged from 0.40 to 0.99 (Table 2).

Discussion
The choice of management action, including no action, for a species in peril must be made carefully with knowledge of the costs and benefits to each option (Baxter et al. 2006). PVA may guide decisions by recommending one management strategy over others based on the projected increase in a viability metric that may result from each proposed action (Possingham et al. 1993). Such relative predictions have been shown to be accurate and robust to uncertainty (McCarthy et al. 2003). However, many conservation practitioners find that the results of models are poorly communicated or are difficult to interpret relative to the costs of enacting proposed management (Borowski and Hare 2007;Addison et al. 2013). We illustrate here how a simple step after a PVA is performed may help clarify the management implications of model results. Our approach explicitly recognizes that real-life management is a one-shot occurrence, and thus it is more intuitive to visualize the outcome of management actions as probabilities.
We highlight three probabilities that should be imperative for developing conservation decisions. First is the probability that a suggested management action will successfully move a population toward the target viability goal. This information is what one thinks of when considering the recommendations of PVAs. It answers the question of how likely it is that expending monetary and political effort to enact a management action will in fact result in the species of concern realizing a positive benefit. Of the 14 PVAs we explored, this outcome was surprisingly rare, typically occurring in fewer than 30% of the realizations, and as low as 15% in some cases. This result suggests that management recommendations from the PVAs

Success
Not Needed Failure Figure 2 For the 14 population viability analyses (PVAs) we evaluated, we tallied the number of the following outcomes: (1) the probability that growth rate was increased from below the target value to at least the target value (white areas, Success), (2) the probability that the growth rate was above the target value before management was enacted (gray areas, Not Needed), and (3) the probability that the growth rate remained below the target after management was incorporated (black areas, Failure).   (Zambrano et al. 2007), (B) Success (Zhang & Zheng 2007), and (C) Perlut et al. 2008b (Not Needed) given by our model. The target λ used for each was 1, represented by the vertical dotted line.
we examined did not stem from model realizations where population growth rates reach a target value, but instead were driven by specific cases in which management was not needed to push the population to its target. We recognize that a sample of 14 published PVA models does not reflect the overall prevalence of this result, however, our method clearly serves to make the occurrence of such results explicit in the context of conservation decisions.
Second, we highlight the importance of calculating the probability that a population already demonstrates a growth rate at or above the target. This probability seems counter-intuitive at first since a species may not be the center of conservation attention if it is not truly in decline. However, a population can show a mean λ below a target and still have some model realizations where λ above the target if either (1) the mean is not far below the target (similar to a pseudo-sink; Watkinson & Sutherland 1995) and/or (2) the variation around the mean λ is high. Our approach explicitly recognizes these two possibilities, and provides clear, actionable information about how this should influence management recommendations.
The mean λ will likely increase if outcomes Success and Not Needed make up the majority of the realizations in a PVA; these are all realizations in which λ met or exceeded the target. In nearly all of the 14 PVAs we evaluated, these two outcomes comprised more than 50% of the realizations, and thus together they pushed the mean λ high enough to suggest that management should be enacted. What we show is that the increase in mean λ was at least as influenced by the outcome Not Needed as it was by the outcome Success. Failure to distinguish between these outcomes in decision-making may lead to large financial expenditures with unsatisfactory conservation results. For example, in some situations a pronouncement of success may not depend on how a viability target was achieved but rather that the target was achieved. Our method allows one to calculate this probability. However, our method also allows explicit calculation of the probability that a population will achieve a target viability goal due to the influence of management alone. When a population is expected to achieve the target goal without management with a high probability, a "do-nothing" approach may be a viable option, particularly when the management action recommended is very expensive. On the other hand, if the recommended management action is inexpensive, a policy-maker may choose to enact the action to increase the chance of reaching the target, even if it is only by a slight margin. Thus, the decision to enact proposed management depends critically on the associated costs, and the willingness to bear those costs by 196 effected parties, given the projected reward in achieving a target conservation goal. Our method makes this tradeoff explicit within the context of PVA.
Third, our approach allows the calculation of how likely it is that management actions are enacted but with the population never attaining the chosen viability target value. This probability is probably the most useful output of our approach since it informs decision-makers of the chance that they will expend limited capital and yet have no evidence that these actions achieved the conservation goal for the threatened species. The degree to which we see this across the 14 PVAs we evaluated varied considerably. As with the outcome "Success," the prevalence of this outcome is determined in large part by how much the suggested management action could influence λ in an absolute sense, and how much uncertainty there was in the system. Policy may impose specific conservation targets for viability metrics. Considering the shift in the distribution of a viability metric both before and after management (Figures 1 and 3) as is done in current sensitivity analyses, may provide information on how much the viability metric can be moved given the resources available. When used in concert with our method, the manager may better understand the probability of achieving a specific goal, given the shift in the distribution that is possible.
We suggest that our approach allows decision-makers to clearly quantify and understand the risks and rewards associated with any particular management action. This method is not intended to replace existing approaches to PVA, but instead seeks to improve its interpretability and utility in decision-making. There are established methods that aid in decoupling the effects of stochasticity versus management (Fox & Kendall 2002;McGowan et al. 2011;Heard et al. 2013). Combined with such approaches, our method can provide a quantitative tool to determine the probability of different management actions being responsible for achieving a target goal. The necessity of such an improvement to PVA is demonstrated by the fact that, within the 14 PVAs tested, the best management solution often had a low probability of success or was actually not required to meet a specific target. Our method should of course be further tested against a larger number of studies and other viability measures, as well as against the actual outcomes of actions taken, to determine how well it can improve the capacity to choose, or predict, the outcomes of conservation actions.