Linking species distribution models with structured expert elicitation for predicting management effectiveness

Effective biodiversity conservation requires robust and transparent prioritization of management actions. However, this is often hampered by a lack of spatially‐explicit data on habitat variables and empirical data on the effect of management actions. Although approaches exist that integrate structured expert elicitation (SEE) with species distribution models (SDMs) to encode species responses across habitat gradients, difficulties remain in predicting management outcomes under different settings, at a region‐wide scale when key habitat covariates are not spatially explicit. Therefore, we developed an approach to integrate SDMs with SEE to capture expert understanding of likely outcomes of management actions for individual frog species, and use this to spatially predict the effect of management actions. We demonstrate our approach across approximately 4000 wetlands in greater Melbourne, Victoria, Australia. As a measure of management effectiveness, we used the change in predicted probability of occurrence of seven frog species at wetlands 10 years after conservation actions are implemented (or not implemented). Management effect was elicited from experts under six scenarios. Individual expert estimates were aggregated using generalized linear models that were then used to spatially predict expected management effects, and a measure of uncertainty in the prediction, at all wetlands. Predicted management effect was strongly influenced by species initial probability of occurrence, with enhancing aquatic and surrounding vegetation an effective action for most species. We discuss practical challenges and recommend solutions in the integration of SDMs and SEE for the spatial prediction of management effect.


| INTRODUCTION
Developing and evaluating regional-scale biodiversity management solutions is challenging but necessary.The funds available are insufficient to address the threats facing biodiversity (McCarthy et al., 2012;Wintle et al., 2019); therefore, robust and efficient approaches to resource allocation are required.This requires defensible estimation of benefit (Evans et al., 2015;Maron et al., 2013) and evaluation of the effectiveness of alternative actions (Bottrill et al., 2008;Ferraro & Pattanayak 2006;Maron et al., 2013).While acknowledging the importance of the costs associated with actions, for example, in maximizing cost-effectiveness of investment (Armsworth, 2014;Kujala et al., 2018;Murdoch et al., 2007), this study focuses on predicting management effectiveness when data are limited.
Species distribution models (SDMs) are used to predict the distribution of a species across geographic space based on statistical relationships between species occurrence and environmental predictors.They predict the probability of occurrence or relative environmental suitability of a species at a location.Predictors can include abiotic variables known to be important for structuring the distributions of species, such as temperature and precipitation, as well as ecologically relevant biotic variables (sensu Austin, 2002).For conservation purposes, however, ecologically relevant variables would ideally include components of the environment that are amenable to management.For example, aquatic vegetation cover and diversity often have a strong influence on amphibian species occupancy and can be directly manipulated through restoration actions (Canessa & Parris 2013;Hamer & Parris 2011;Heard et al., 2013).When such predictors are included, SDMs can be used to predict the effect of management and provide decision support to managers, conditional on understanding the relationship between management and environmental variables of interest.Unfortunately, spatially explicit data on such fine-scale habitat predictors are often lacking, and the relationship between management actions and variables of interest can be uncertain.For instance, while aquatic vegetation cover and diversity are commonly measured at sites, at the time of survey, such data are rarely available for unsurveyed sites at regional scales.Consequently, they are rarely incorporated as predictors in frog SDMs despite their ecological importance and management relevance.
Expert elicitation offers an approach for augmenting understanding and estimation of the effect (benefit/disbenefit) of management actions that cannot be directly incorporated into SDMs.Experts can hold a rich understanding of species and ecosystems, which if carefully elicited, can be used as a form of data to inform predictions.Defined as a formal process for obtaining and aggregating judgments from experts, expert elicitation can also provide proactive decision support in a timely and inexpensive manner (Kuhnert et al., 2010;McBride & Burgman, 2012).
While a variety of environmental models have been created based on knowledge elicited from experts (Krueger et al., 2012), the quality of elicited estimates depends on the scientific rigor of the elicitation method and its ability to improve judgments (Martin et al., 2012).Like anyone, an expert is prone to psychological biases, which can influence the output of the elicitation (Martin et al., 2012).These frailties have been outlined and discussed in the literature thoroughly (e.g., Martin et al., 2012;McBride & Burgman 2012).Fortunately, there are methods that can reduce the effect of biases and heuristics (Burgman, 2005;Dalkey & Helmer, 1963;Gigerenzer, 1996;Hanea et al., 2017;Speirs-Bridge et al., 2010).In particular, formal structured expert elicitation (SEE), such as the IDEA protocol (Investigate, Discuss, Estimate, Aggregate; Hanea et al., 2017;Hemming et al., 2018), provide an explicit framework, drawing on foundations from mathematics, decision-theory, and psychology, to improve accuracy while minimizing the effects of important biases (O'Hagan et al., 2006).They follow transparent methodologies, are well documented and provide options for empirical evaluation and validation (Hanea et al., 2018).
An important consideration in expert elicitation is how to aggregate judgments when there is more than one expert (Hemming et al., 2020).The IDEA protocol uses mathematical aggregation, where individual judgments are combined through a mechanistic rule, such as opinion pools (e.g., averaging) (Soares et al., 2018).Another mathematical option, rarely undertaken in SEE (but see Garthwaite et al., 2008), involves using individual expert estimates as separate data points in the creation of statistical models.This allows between-expert variability around a quantile (e.g., in terms of regressors) to be accounted for as well as within-expert uncertainty around the median.
In this study, we describe a set of methods through application to a case study in which SDMs and SEE are integrated to enable region-wide prediction of management effects.Individual SDMs are used to estimate initial probability of occurrence of seven frog species at waterbodies across our study area.We design and conduct SEE to predict changes in probability of occurrence under management and do-nothing scenarios, accounting for the influence of initial probability of occurrence.We build on methods for aggregating estimates from experts by using experts as samples and fitting generalized linear models (GLMs) to model effects of multiple management actions on each species in wetlands across greater Melbourne.Practical challenges and recommended solutions, in the integration of SDMs and SEE for the spatial prediction of management effect are identified.
The study area was the greater Melbourne region ($13,000 km 2 ) located in Victoria, Australia.It encompassed five major catchments that hold about 65,000 waterbodies, including farm dams (Figure 1).The waterway manager of the region was Melbourne Water, a statutory corporation owned by the Victorian State Government.Greater Melbourne had a population of about 4.5 million people (Australian Bureau of Statistics, 2016), spanned about 3000 km 2 of urban and peri-urban space, with the remaining area containing rural and agricultural land, intact closed water supply catchments, and forest.
This study focused on wetlands over which Melbourne Water had some responsibility or management influence ($4000).With frogs as the target biota for the study, we excluded saline wetlands and estuaries.Seven wetlanddependent native frog species were considered, for which we had sufficient presence-absence survey data available to develop SDMs: Crinia signifera, Limnodynastes dumerilii, Limnodynastes peronii, Limnodynastes tasmaniensis, Litoria ewingii, Litoria raniformis, and Litoria verreauxii.Details of their conservation status and main threats are summarized in Supplementary Information 1.

| Conceptual framework
The process undertaken to understand and predict benefits to frogs of alternative management actions in the study involved: (1) using SDMs (previously developed) to predict initial probability of occurrence of the seven species across the region; (2) identification of candidate actions; (3) SEE to derive the expected effect of candidate actions at different levels of initial species probability of occurrence; (4) developing statistical models for each species incorporating the initial probability of occurrence (from 1), as an explanatory variable, and derived effect (from 3), to predict spatially explicit effects across the study region; and, (5) estimation and ranking the outcomes of actions at each site.

| Species distribution models
SDMs (Generalized Additive Models; Hastie & Tibshirani, 1990) for each species were fitted using existing (non-repeat visit) presence-absence data collected over 10 years in the study region (Supplementary Information 2).These were used to predict the initial probability of occurrence of each species at each wetland site.Model development and predictive performance evaluation followed The study area sits on the southern border of Victoria, Australia, includes five catchments that hold approximately 65,000 wetlands (including dams) (Supplementary Information 2 for details of mapping data).The region includes urban, peri-urban, and rural areas as well as intact closed water catchments.methods outlined in Wintle et al. (2005).Based on receiver operating characteristic curve (ROC), (Hanley & McNeil 1982) and Miller's calibration slope (MCS), (Miller et al., 1991), there was evidence that models for each of the seven species showed reasonable ability to discriminate between presence and absence sites and to correctly predict the proportion of sites occupied given a certain environmental profile (ROC ≥ 0.73; MCS = 0.99-1.11).A summary of the SDMs is provided in Supplementary Information 2 with details on the data used and method of model development and evaluation.

| Candidate management actions
Relevant and suitable candidate actions were identified through two workshops with waterway managers at Melbourne Water, consultation with frog experts, and a review of the published and gray literature.The final selection of actions (Table 1) was primarily based on key threats to the species as well as identification of actions that Melbourne Water and other relevant stakeholders would consider implementing.The selection included five management actions (or action-sets) and a donothing scenario that serves as a baseline for comparison.

| Expert elicitation
The parameter elicited (Section 2.5.2) was the expected change in frog probability of occurrence after a 10-year timeframe, given an initial probability of occurrence value and implementation of an action.This was used to calculate the species-level action effect under alternative scenarios: where the effect (benefit/disbenefit), B, of action, a, to species, j, is the difference between probability of occurrence under an action scenario, AP, and probability of occurrence under a do-nothing scenario, NP.Ten years was used because this reflects the timeframe of the strategy objectives set by Melbourne Water (2018).Parameters were elicited at two levels of initial probability of occurrence, P 0 : 0.10 and 0.95.Although more than two levels would be preferred in characterizing the relationship between initial probability of occurrence and management response, this would have substantially increased the number of questions experts would be required to answer due to the combination of actions, species, and wetland types (i.e., 42 questions per level).
We employed the IDEA protocol (Hanea et al., 2017;Hemming et al., 2018), which has been shown to improve expert judgments relative to simpler ad-hoc approaches, such as relying on a single expert (Burgman et al., 2011;Hanea et al., 2018;Hemming et al., 2018).It involves a three-stage process: (i) pre-elicitation (questionnaire design and recruitment of experts); (ii) elicitation (involving two rounds of independent expert estimates with a review of group responses and anonymous discussion in between), and (iii) post-elicitation (aggregation of expert estimates) (Figure 2).We built on this approach by including an additional step of aggregation to develop spatially predictive models of species-action effect.The application of the approach to this study is described below, with expanded details provided in Supplementary Information 3.
T A B L E 1 The five candidate management actions, their means objective(s) related to the common fundamental objective of species persistence, and their descriptions (with detailed descriptions in Supplementary Information 3).

Management action and objective Description
Do-nothing scenario Objective: None Not managing habitat or threats related to the species at a wetland site.

| Pre-elicitation
Elicitation surveys were developed to derive estimates of the change in probability of occurrence under alternative action scenarios.They provided a description of the study area, candidate actions, type of wetlands, and species habitat requirements and threats (Supplementary Information 4).The four-point format of questioning (Hemming et al., 2018;Speirs-Bridge et al., 2010) was presented to obtain a lower bound, best estimate, upper bound, and confidence level per question for the response of the focal frog species to each action.We adopted direct elicitation in asking the experts for estimates in terms of frequencies, as this improves communication and understanding in elicitation studies (Gigerenzer, 1996).For example, "How many wetlands out of 100 would be occupied in ten years time if …?".We approached 24 experts, recruited 10 participants and seven completed the whole elicitation stage.The number of species (seven), varying starting occupancies (two), and actions (six, including the do-nothing scenario) created 84 questions per expert.To assist participants in evaluating such a complex problem, we narrowed their scope by allocating each expert to one of two wetland types that Melbourne Water manages: stormwater wetlands (four experts) or conservation/ natural wetlands (three experts).

| Elicitation
The elicitation stage ran remotely over 6 weeks between March and April 2018 (Figure 2).Surveys were distributed via email with a completion timeline of a week.The discussion step was initiated through distribution of a summary report to participants containing a summary of individual and aggregated expert estimates (via quantile averaging) with standardized confidence intervals (to 90%).Comments were encouraged through the use of participant code names and we provided any clarifications as required.Subsequent revised participant estimates were summarized again in a report circulated to experts as a final opportunity to review and clarify any misunderstandings.

| Post-elicitation
Following elicitation we used individual expert estimates, rather than the averages, to develop GLMs for predicting the effect of each action to allow between-expert variability to be carried through the analysis.This was required to support internal consistency of expert judgments on a species response per management action.Prior to building the GLMs, a new dataset was created by fitting a beta distribution to the three expert judgments of best estimate and credible intervals (to consistently capture the within expert variability) using R (R Core Team 2017).For each expertspecies-action-initial occurrence combination, values were extracted from the fitted beta distributions for the 5th, 50th, and 95th percentile (representing the lower bound, best, and upper bound estimate).The percentile values under the do-nothing scenario were subtracted from corresponding values of the action scenario, to give the action effect (B) for that combination.These steps are summarized in Figure 3 with further details and example R code provided in Supplementary Information 5.
A separate GLM with Gaussian error structure was then created for each of the extracted 5th, 50th, and 95th percentile effect values for each action-species-initial occurrence combination to represent uncertainty around the best estimate.For each action and species, the models for action effect were defined as either Equations (2a) or (2b).
where P 0 is the initial probability of occurrence [0,1] from the original SDMs, A is the action type (categorical) and W is the wetland type (categorical).As we were mainly concerned with predicting benefit of actions spatially, wetland type was not included in the model unless it improved the accuracy of predictions of action effect.
The key indicator of model performance used to assess relative suitability and final model selection for each scenario was Akaike's information criterion (AIC), (Akaike 1973).The final model for each action-species combination was used to predict the effect of each action to each species at each wetland in the study area, under the best estimate case, the lower bound case, and the upper bound case.Details of how the models were developed and assessed are shown in Supplementary Information 6.
Actions were independently ranked at the wetland scale through calculation of the overall action effect: where AP ij is the probability of occurrence of species, j, at wetland, i, given action, a, and NP ij is the probabilty of F I G U R E 3 Beta distributions were fitted to the standardized data for each expert-species-initial occurrence combination using the lower bound, best, and upper bound estimates.This was done separately for the action (AP) and no-action (NP) scenarios (the first and second plot, respectively).After fitting, random samples from the two fitted distributions were drawn.The difference between these samples was used to derive the action effect, that is to say, for each expert-species-initial probability of occurrence combination; the values under the donothing scenario were subtracted from those of the action scenario, to give the empirical distribution of action effect (B) for that combination (the third plot).The 5th, 50th, and 95th percentiles of this empirical distribution were then estimated from the samples.A generalized linear model was developed such that B aj $ P 0.i + A + W, where initial probability of occurrence (P 0 ), action type (A), and wetland type (W) become explanatory variables of the effect (B) of action, a, for species, j, at wetland, i.
occupany given the do Ànothing scenario.The best action to implement, α i , for each wetland, i, is defined as the action, a, with the maximum value among O ai on a 1 , …, a 5 .

| Elicitation
Three of ten recruited participants withdrew from the study early in the elicitation stage, citing conflicting time commitments, and two of the remaining experts also commented that the surveys took a long time.This suggests that the survey design was pushing the limits of acceptable voluntary time for experts and that any additional questions could increase cognitive fatigue and risk further participant dropout.About one-third of participants provided comments during the discussion stages, which facilitated clarifications.Accordingly, approximately half the participants updated their estimates following the discussion phase.
The expert estimates elicited for each species-action scenario are summarized in Figure 4, where the 5th, 50th, and 95th percentiles extracted from the beta distributions (per expert) have been averaged.The maximum predicted effect was an increase of 0.19 relative to the donothing scenario (L.raniformis under Action 5, Do All).Plausible outcomes ranged from À0.24 (L.raniformis and Action 1, Enhance surrounding vegetation) to 0.42 (also L. raniformis but for Action 5, Do All).Therefore, the worst predicted outcome for L. raniformis involved a 24% reduction in probability of occurrence under Action 1, relative to the do-nothing scenario, while the best predicted outcome involved an increase of 42% under Action 5.In addition to Action 5, Action 2 (Enhance surrounding and aquatic vegetation) tends to be beneficial for most species, whereas, Action 4 (Construct a wetland) appears more variable across species and initial probability of occurrence levels in its effect (Figure 4).
As demonstrated by the lower bounds (Figure 4), it is possible to have a negative effect under each of the candidate actions.However, when starting out with a probability of occurrence of 0.10 at a wetland, expert assessment Expected effect of candidate actions on individual species relative to the do-nothing scenario, as derived from expert estimates of changes in wetland probability of occurrence when initial probability of occurrence is 0.10 (dots) and 0.95 (triangles).Dots and triangles represent the median with the error bars and the credible interval, standardized to 90%.For each species-action and initial probability of occurrence scenario, the 5th, 50th, and 95th percentiles extracted from the beta distributions (per expert) were averaged.However, individual estimates were used in the creation of generalized linear models to predict effectiveness spatially.
suggests a 90% probability that the action "Do All" will be beneficial for more than half of the species (Figure 4b-d,f).For five out of seven species, the effect, based on the 50th percentile, is generally greater when starting out with a probability of occurrence of 0.10 compared to 0.95.

| Post-elicitation
The final predictive models developed and selected for each species are shown in Table 2. Initial probability of occurrence and wetland type had a significant influence on management effect in 57% and 52%, respectively, of scenarios ( p < .05).Three out of 21 models performed worse according to the AIC than the null model (C.signifera 5th percentile, L. ewingii 50th percentile, L. ewingii 95th percentile).
The initial probability of occurrence at which greatest effect was predicted and the direction of change in effect varied between species and depended on which percentile is used to generate the models (examples shown in Figure 5).However, the patterns found for one action (within a species) are generally the same for all actions.
In most cases, for predictions using models based on the experts' best estimate (50th percentile), action effect decreased with increasing initial probability of occurrence (exceptions: L. raniformis and L. verreauxii).For most species (e.g., L. peronii, Figure 5), the point at which greatest effect can be expected (based on the 50th percentile) is when the uncertainty interval and initial probability of occurrence are the lowest.However, for L. raniformis, T A B L E 2 Models developed and applied to predict effect, B, of each action for each species at wetlands using the experts best estimate (50th percentile, first row) and credible interval (5th and 95th percentile, 2nd and 3rd row, respectively); their Akaike information criterion (AIC, Akaike 1973); reduction in deviance; and the variables and variable-levels shown to have a significant influence on predicted benefit.a highly threatened species, greatest benefit (based on the 50th percentile) can be expected where uncertainty and initial probability of occurrence are highest (e.g., L. raniformis, Figure 5).Actions that consistently showed more positive impacts were Action 2 (Enhance surrounding and aquatic habitat) and Action 5 (Do All).The highest ranked action (the best action, α i ) was Action 5 (Do All) at all wetlands, regardless of uncertainty (represented by the 5th and 95th percentile), wetland type, and initial probability of occurrence.With a maximum difference between species of 0.1 in the predicted change in probability of occurrence under Action 5, no species would be particularly worse or better off.This may be because different frogs have similar habitat preferences with respect to the actions we included and/or that Action 5 covers many types of habitat restoration needs.

| DISCUSSION
We developed and applied an approach to predict management effectiveness across region-wide scales when spatial data on management-relevant habitat variables is incomplete.Our approach provides a transparent and tractable way to integrate SDMs and expert knowledge to provide estimates of benefit that could be used to prioritize management actions.Outputs from this approach can inform decision-makers about which actions (or action-sets) are likely to be best for individual species on a wetland and regional scale and can serve as an important preliminary step to a regional spatial prioritization for planning and strategy development.

| Methodological developments
Building on a SEE approach (the IDEA protocol), we offer an alternative to the way expert estimates are typically aggregated.Using GLMs to model multiple expert judgments about the influence of actions provides a simple and technically coherent means of aggregating judgment into a single prediction that can be used for prioritization.Along with producing separate GLMs for experts' best estimate and credible interval, this allows both between-and within-expert variability to be accounted for.This is the first application of the IDEA approach to utilize this type of aggregation and could be considered an alternative to linear opinion pools of fitted distributions (Clemen & Winkler 2007).
To our knowledge, there are no studies that elicit from experts the expected benefit of conservation actions that explicitly consider the starting point in the measure of interest.Yet, we found that the expected effect of an action varies with initial probability of occurrence considered (Figure 4, Table 2), highlighting the need to account for the starting value in the performance measure for management effect or to make assumptions about it explicit.For most species and actions, the action effect based on the expert's best estimate is greater when starting out with a probability of occurrence of 0.10 compared to 0.95.This may be because there is much more room to improve when starting with a low probability of occurrence than when initial probability of occurrence is already high.
Given that the initial probability occurrence of a species influences the predicted management effect, it is important that the predictive accuracy of SDMs relied F I G U R E 5 Examples of predicted benefit (y-axis) of actions (in columns) to each species (in rows) with increasing starting probability of occurrence (x-axis).Upper, middle, and lower lines represent the predicted upper bound of the expert's credible interval, the median, and the lower bound.Uncertainty (credible interval) in the benefit of an action tends to be the lowest, and benefit (based on the median) tends to be the highest, at lower starting probability of occurrence (e.g., Limnodynastes peronii), although there were exceptions (e.g., Litoria raniformis).
upon in applications of this approach are as strong as possible.Otherwise, the prediction of effect may be unreliable.While the SDMs used were found to have good predictive accuracy (Supplementary Information 2), measures to improve on this include using larger sets of unbiasedly sampled occurrence data for model training, more proximate predictors and independent data for more rigorous performance evaluation and validation.

| Application to the case study
The elicitation results (Figure 4), focusing on the experts' best estimate (50th percentile), showed that the most beneficial action for all species was, unsurprisingly, Action 5, Do All, with Action 2, Enhance surrounding and aquatic vegetation, usually the second-most beneficial action.Interestingly, Action 1, which is the same as Action 2 but without the aquatic vegetation component, was almost always the least beneficial action, suggesting a marked difference in the benefit of enhancing aquatic vegetation habitat relative to surrounding terrestrial vegetation habitat.Previous research has highlighted the importance of aquatic vegetation cover and diversity for frog species richness and occurrence (Hazell et al., 2001(Hazell et al., , 2004;;Heard et al., 2008) and population viability (Heard et al., 2013).Our findings reflect this; yet, there is limited reference to protecting and enhancing aquatic vegetation for frogs in the performance objectives of Melbourne's waterway management strategies (Melbourne Water 2013Water , 2018)).
Although initial probability of occurrence influenced expected management effect of individual species (Figures 4  and 5), when it came to identifying the best action (greatest summed effect across species) at each site, initial probability of occurrence made no difference to the ranking order.Nevertheless, the effect size of an action is still important to assess because the action with the greatest summed benefit may not be a preferred option if one or more of the species is at risk of becoming locally extinct under that scenario or if the summed benefit is not much greater than the next best option (relative to cost).

| Identifying the best locationspecific action
In identifying the best action, we have used a rather narrow benefits-only definition of "best," however, we recognize that there are other important factors, such as costs and complementarity that ought to, and can, be accounted for (e.g., Auerbach et al., 2014;Chadés et al., 2015;Leathwick et al., 2010).In particular, management costs are important to incorporate before decisions are made so that the return on investment can be maximized and because ignoring them can lead to investment in expensive options and missed opportunities (Ando et al., 1998;Balmford et al., 2000;Wilson et al., 2007).Indeed, implementing all actions at wetlands (Action 5) will almost certainly be expensive and, if the marginal benefit between Action 5 and Action 2 is minimal, more species and their habitats could probably be conserved for equivalent funds under Action 2 or a combination.

| Key challenges and recommended solutions
The primary practical challenge in integrating SEE with SDMs to predict management effect was the trade-off between study complexity (number of actions, species, and habitat types) and sample size.The number of survey questions (84 in total; 1 week given per 42 questions) pushed the limits of a voluntary and remote elicitation arrangement, with three participants dropping out early on.As noted previously, too many questions can impact fatigue and estimate accuracy (Johnson et al., 2012;Spetzler & Stael von Holstein 1975).This resulted in a small sample size (3-4 experts) per scenario, when, ideally, 5-12 experts per scenario would be recruited to ensure expert diversity.Possible solutions for future research include recruiting more experts so that questions can be shared, increasing the time period over which the elicitation was conducted, reducing the number of actions to be considered, convening experts as part of a working group and/or, offering reimbursement for expert time.None of these options was appropriate under the constraints of the study.
There were several additional limitations of the current study.We chose to elicit estimates for just two values of initial probability of occurrence (per action-species combination) and assumed a linear relationship, however, different functional forms for the influence of initial probability of occurrence on management are possible.Yet, to encode the functional form, at least three values of initial probability of occurrence would need to be used, creating another 42 questions.In addition, we used just two wetland types in our study design (conservation/natural and stormwater management wetlands) while wetlands are far more diverse.Wetland type would influence expected effects and accounting for it could improve accuracy and uncertainty of predictions.
Therefore, additional future research recommendations include modifying the survey design to allow more wetland types and at least three levels of initial probability of occurrence, while minimizing the risk of expert fatigue associated with more questions.Finally, developing an empirical evidence base of measured on-ground management effects of the candidate management actions will be important for testing and validating expert elicited predictions under a range of conditions of interest.

| CONCLUSION
Our approach to SEE and aggregation of expert estimates allows integration with SDMs and prediction of benefit spatially.It is widely applicable to many problems where there is a pressing need to prioritize resources regionally, an absence of empirical data on management effect and of spatial data on key habitat variables, but SDMs (or the data to build them) are readily available.While SDMs can be validated, there are often challenges with crossvalidating expert estimates due to the number of available experts.However, this approach allows one to account for both within and between expert uncertainties in predictions of management effect.It offers insight for decision-makers on how to understand and maximize benefit across species on a site and regional scale while minimizing risks of negative effects.

F
I G U R E 2 The three stages of the elicitation approach (adapted fromHemming et al., 2018): pre-elicitation (identification of candidate actions, preparation of surveys and recruiting and briefing experts), elicitation (two rounds of independent expert estimates) and postelicitation (aggregation of expert estimates).During the elicitation stage, experts are encouraged to: investigate the problem in providing a private individual estimate in the first round of responses; discuss results anonymously; and estimate a second round of responses (an option to update previous estimates following the discuss step).