Principles for the production of evidence-based guidance for conservation actions

Many types of guidance documents inform conservation by providing practical recommendations for the management of species and habitats. highlight uncertainty in the key evidence that supported the recommendations. These deficiencies can lead to misguided and ineffective conservation practices, policies, and decisions, and a waste of resources. Based on this review and co-design by experts from 14 organizations, we present a set of principles for ensuring sufficient and relevant evidence is transparently incorporated into future conservation guidance. Producing evidence-based guidance in line with these principles would enable more effective conservation outcomes.

highlight uncertainty in the key evidence that supported the recommendations.
These deficiencies can lead to misguided and ineffective conservation practices, policies, and decisions, and a waste of resources. Based on this review and codesign by experts from 14 organizations, we present a set of principles for ensuring sufficient and relevant evidence is transparently incorporated into future conservation guidance. Producing evidence-based guidance in line with these principles would enable more effective conservation outcomes.

K E Y W O R D S
evidence-based conservation, knowledge, policy, practice, recommendations 1 | INTRODUCTION Conservation practitioners must decide what actions to implement to achieve their objectives. Despite aiming to make the most appropriate decisions, practitioners are often seriously constrained by time and resources (James et al., 2001;WWF & Credit Suisse, 2014). Comprehensive searching, reading, and interpretation of the scientific literature is often impractical due to a number of factors including: the volume of research; publication, spatial, and taxonomic biases in the evidence base (Christie et al., 2020;Pullin & Knight, 2005); barriers to accessing information such as pay walls; low generalizability of studies; lack of time to devote to such efforts; and the variable quality and reliability of studies .
For many practitioners, conservation guidance of various sorts is deemed to be a practical solution that provides consolidated advice about what works and how to do it, without having to survey the primary scientific literature (Brancalion & Holl, 2020). Here, we define guidance as "an authoritative source of information and recommendations with the objective of informing the decisions and actions of practitioners." Typical guidance sources include books (Ausden, 2007;Sutherland & Hill, 1995), reports, guidance notes, leaflets, and online material.
Guidance in conservation covers a wide range of topics from single species management to ecosystem restoration, as well as best practices for surveying and monitoring. Guidance may also include instruction for adopting an approach that encourages the user to make sequential decisions about which evidence-based approach is most appropriate for their context (The Woodland Trust, 2021). Adhering to guidance may be an integral part of the licensing process for infrastructure developments or other aspects of regulatory compliance, in particular when working with legally protected species or management of critical habitats (English Nature, 2001;Langton et al., 2001). An important aspect of effective guidance is that it relies on scientific evidence to inform its content and recommendations. Evidence can include documented evidence such as peer-reviewed scientific literature and gray literature. This evidence should be sourced, utilized, and referenced alongside other sources of evidence including experience and expert opinion (including local and Indigenous knowledge) (O'Brien et al., 2020) to advise on values, resources, practices, and costs. When we refer below to "evidence," we mean documented scientific literature.
The driver for our work is concern about the quality of guidance currently in circulation. We examined the inclusion of evidence in conservation guidance in the United Kingdom and Ireland to understand the level of transparency in the following: methods of production, source of information underlying the recommendations, date of the source material used to write the guidance, and the reporting of uncertainty in the recommendations based on the strength of the evidence. Although wellformulated and presented guidance can result in the effective translation of evidence into practice (Walsh et al., 2015), poorly formulated guidance can lead to misdirection or misguided decisions, resource waste and could even do more harm than good (Hunter et al., 2021;Pullin & Knight, 2005).
We suggest the development of conservation guidance could draw from the structure employed in medicine, where guidance is routinely grounded within verifiable evidence. For example, in the UK, information guidelines are produced and regularly updated by the National Institute for Health and Care Excellence (NICE) covering topics from antimicrobial prescribing to social care, with recommendations based on the evidence for the effectiveness of actions, as well as their side effects and costs (Melnyk & Fineout-Overholt, 2011). Physicians and other healthcare professionals are expected to combine the lessons provided in guidance with their individual clinical expertise on a case-by-case basis, noting that even the best available scientific evidence needs to be tailored to individual patients and situations that may have specific risk factors for side effects or complications (Rycroft-Malone et al., 2004). Medicine has a well-funded and supported collection of organizations that synthesize research, produce guidance, and make it accessible to practitioners (e.g., Cochrane and NICE). Conservation may benefit from adopting these regulatory approaches and institutional support structures (Kadykalo, Buxton, et al., 2021).
In this paper, we examine the presentation and referencing of evidence in the production of conservation guidance on mitigation and management of species and habitats in the UK. Based on our findings, we propose a set of principles to improve the frequency, quality, reliability, and transparency of evidence used in guidance for conservation. They build upon other established principles for guidance creation, such as how to define the scope and purpose of guidance, and managing stakeholder participation and consultation (CIEEM, 2021).

| CURRENT GUIDANCE ON CONSERVATION ACTIONS IN THE UK AND IRELAND
Guidance documents are one commonly used source of advice (Cook et al., 2010(Cook et al., , 2012Seavy & Howell, 2010;Young & Van Aarde, 2011) which reach a broad range of practitioners and are frequently relied upon to inform decision-making. Studies have shown that scientific advice is highly valued by practitioners when making these decisions (Walsh et al., 2019). Despite this, it remains unclear how well the scientific evidence has been incorporated into commonly used guidance documents.
In the last decade, the source of evidence for conservation guidance has become increasingly accessible through open access journals, improved search engines, and open access databases. These resources provide critically appraised and synthesized scientific evidence and make the production of evidence-based guidance attainable for more organizations. However, there has been little apparent attempt to ensure that conservation guidance is routinely and clearly linked to the available scientific and technical evidence, unlike in medicine.

| SEARCH METHOD
During February-March 2020, we conducted a search of the available guidance on conservation management and mitigation actions in the UK and Ireland. This involved searching online directories recommended by practitioners (NHBS and CIEEMs Good Practice Guidance List) as well as the websites of conservation non-governmental organizations (NGOs) and governmental organizations based in the UK and Ireland. This search yielded 357 pieces of guidance from 58 different organizations (see Supporting information for list of websites searched). Of these, 56 documents (approx. 16%) were behind paywalls and were unable to be included in the review (Table S1), leaving 301 guidance documents to be assessed.
These 301 documents were reviewed and tagged for several key attributes: year of publication, inclusion of methodology for evidence collation and recommendations (yes/no), reference list (yes/no), in-text citations (yes/no), and uncertainty in recommendations stated (yes/no). Documents with a reference list were then searched for references where the title suggested that the source tested an action. These papers then had their abstract reviewed to confirm the test of an intervention (i.e., a quantitative measure of the effects of that intervention), and in some cases, methods were also checked to confirm this assumption (using methods described in Sutherland et al., 2019). If no papers were found testing the effect of a relevant conservation action it was tagged "no" for correct referencing. If one or more papers testing the effects of a relevant conservation action was found, it was tagged "yes" for correct referencing.
The review of these documents revealed several issues with how evidence is used in the creation of guidance.

| Guidance recommendations not supported by relevant references
Of the 301 documents listed in Table S1, only 29% (n = 87) contained references or gave the sources of information. Of these 87 documents, only 32% (n = 28) referred to a study testing the effectiveness of the recommended actions. Therefore, only 9% of all the guidance reviewed contained a reference that was relevant to its recommended action. Even this may be an overestimate as documents that gave multiple recommendations did not always include references for each recommended action. In these cases, the source of evidence used to make recommendations cannot be identified (they may be based studies that were not referenced) and may leave the recommendation too open to interpretation to be deemed as evidence-based guidance. In cases where documents did not reference appropriate studies about effectiveness of management actions, the references were to other organizations' websites or to studies about the natural history or ecology of species or habitats, or even to other guidance documents. Although evidence on threats, trends, ecology, and so on are all important components of guidance, recommendations for actions are not evidence-based without a transparent, assessable reference to evidence for that action. The 56 documents behind paywalls were manuals and handbooks, which are likely to contain reference lists and may be better referenced, but these could not be accessed to check.

| Lack of information on how evidence was collated
Only four documents (1.3%) included a methodology for the literature search that led to the collation of evidence, such as the search terms and databases used or guidelines on why some materials were included and others excluded (Haddaway et al., 2015). This lack of methodology makes it difficult to judge how robust, repeatable, and unbiased the initial collation of evidence may have been. In turn, this opacity makes updating guidance documents more time-consuming and expensive.

| Guidance is not regularly updated
Of the guidance documents that contained a date, 63% (n = 187) were over 10 years old (from date of search), with some commonly used guidance not updated for decades (English Nature, 2001;Langton et al., 2001). None of the searched documents contained a future revision date. For many species and habitats, it seems likely that new primary evidence has become available since. Guidance based on old evidence may therefore not reflect current knowledge of better practice, and may hinder the user seeking newer alternative approaches. We were able to assess the year of publication for those documents behind paywalls that had not been included in other analyses (n = 56) and found that 71% (n = 40) were also over 10 years old. Therefore, the documents behind paywalls do not appear to have more up-to-date evidence supporting their recommendations than open access documents.

| Unclear strength of the evidence used to make recommendations
Only 5.6% of the documents (n = 17) indicated the reliability of their recommendations. This can lead to recommendations being interpreted as effective despite significant ambiguity (van der Bles et al., 2019). Even those documents that did include some indication of uncertainty only did so for a subset of the recommendations. No document provided strength of evidence for all recommendations.
Seventy one of the guidance documents (24%) were UK government guidance available through gov.uk and none included references or methods. To test whether this government guidance unduly affected the above findings, we removed these articles from the dataset and found that the same issues persisted. Only 38% of the remaining documents contained references, 1.7% included methods, and 7.4% detailed the strength of evidence.
Every reviewed guidance document was affected by one or more of these limitations, and therefore guidance users have serious challenges in interpreting the recommendations. In some cases, guidance may be unavailable (i.e., nothing was published on that topic), out of date, poorly informed, poorly described, based upon invalidated or biased subsets of the evidence or produced using unclear methods, any of which means the user may not be able to effectively assess the quality. These problems demonstrate a need to improve the quality and method of producing conservation guidance. Furthermore, the search for documents demonstrates the fragmented nature of guidance delivery, with multiple organizations producing guidance and no single directory to search for documents or to find out if guidance has been updated, or even exists. If finding guidance is difficult and time-consuming, and many key documents are behind paywalls costing from £5 to £250+, it is possible that the available guidance is not being used in all relevant settings.

| METHODS FOR DEVELOPING EVIDENCE-BASED GUIDANCE FOR CONSERVATION ACTIONS
The authors of this paper are directly involved in the production and use of conservation guidance and are in an informed and experienced position to consider how evidence should be incorporated. Authors were chosen to cover the spectrum of professionals involved in developing and using conservation guidance (scientists, practitioners, guidance writers, and users) and work in diverse fields from species to landscape conservation, evidence synthesis and use, and policy and mitigation, allowing the incorporation of a wide range of opinions on what is important in guidance production. Indigenous knowledge keepers and holders were not part of this process given the UK context and this requires additional consideration where relevant (Kadykalo, Cooke, & Young, 2021).
Based on principles constructed for the creation of evidence-based guidance in medicine (Institute of Medicine, 2011), and through a workshop and multiple rounds of consultation, the authors have produced a set of BOX 1 Principles for evidence-based guidance Collating evidence 1. Scientific evidence should be reviewed and where available incorporated when formulating recommendations.
Review the available scientific evidence on conservation actions (either from peer-reviewed studies, databases, gray literature or expert consultation) and extract key messages to inform the development of recommendations. There are now many databases available that synthesize relevant evidence, such as conservationevidence. com, environmentalevidence.org, and databases that collate gray literature such as Applied Ecology Resources. These can drastically reduce search, reading and interpreting time as well as overcoming access barriers. The evidence should be considered by stakeholders to judge its strength and relevance (Salafsky et al., 2019) and assessed alongside the experience and knowledge of stakeholder groups, which must include relevant experts. The date, search terms, and databases used for searching for evidence should be stated (Haddaway et al., 2015). Non-English language papers should also be considered in the search to avoid bias (Konno et al., 2020).

Conduct repeated searches of the literature regularly and update guidance to include new studies when required.
To ensure that guidance is based on the most up-to-date information, guidance should state when the evidence was searched and set review dates. We suggest reviewing the evidence every 5 years. When critical new information is available, guidance should be updated. Out-of-date guidance should be updated and then archived, with clear links to the updated version provided. If the original evidence synthesis clearly specifies its references and justification for recommendations, then updating the guidance will be easier and faster.

Presentation and interpretation of evidence should be neutral.
The information should be presented factually and objectively and those engaged in collating and synthesizing the evidence should operate as neutral brokers. This can be difficult for some authors or organizations involved in the production of guidance, particularly where there is an advocacy objective or when they have been involved in producing the relevant evidence. It may therefore be beneficial to have guidance peer-reviewed or produced collaboratively across communities of practice, to avoid bias affecting the presentation of the evidence. Some organizations may find it hard to remove all advocacy of their agenda from guidance. Such conflicts of interest should be stated explicitly.

Bias and limitations of the reviewed literature should be stated explicitly.
State the problems (such as publication bias) and uncertainty that is inherent in any study or synthesis. Any potential bias or limitation in evidence searching and collation strategies should also be clear (Dicks et al., 2017). 5. Where possible, assess and report on the cost (financial and other), cost-effectiveness, and side effects of potential interventions. Information on the costs and outcomes on factors other than biodiversity should be collected where possible. This should include possible areas of conflict, for example, with other biodiversity or socioeconomic priorities. This can help inform the recommendation process.
Making recommendations 6. Specify the type and source of evidence used to make recommendations. Make clear what evidence has been used. Document the review process and sources (e.g., scientific papers, gray literature, expert opinion, Indigenous knowledge). Details of methods should be provided either in the guidance document or in a linked source (e.g., weblink or QR code) that explains how the evidence was identified and extracted. This allows the details of the original studies to be available to those that are interested to research further.
7. The strength of the evidence behind recommendations should be transparent. If there is uncertain or conflicting evidence this should be made apparent, either by explicitly describing the evidence or using appropriate terms (strong evidence, some evidence, weak evidence, studies predominantly support, etc.; see Table 1). The scale of inference should also be clear, such as if the evidence is based on a subset of conditions or varies with context (e.g., species, location).
principles for the production of feasible, rigorous, and reliable guidance. This review focuses on guidance for the delivery of conservation actions (e.g., management and mitigation), but the principles could be applied to guidance covering other aspects such surveying and monitoring.

| HOW EVIDENCE-BASED GUIDANCE SHOULD BE PRESENTED
Box 1 lists nine principles to ensure evidence is correctly and transparently used in the production of guidance.
Not all guidance producers will be able to achieve all principles immediately, and their upcoming documents may still contain useful information. However, we recommend that guidance producers strive to ensure that new or updated guidance meets these principles.

| THE CREATION OF EVIDENCE-BASED GUIDANCE DOCUMENTS
High-quality guidance does not dictate a universal approach to conservation actions but supports decisionmaking by clearly describing the available evidence and recommendations. To make tailored decisions, users need to be able to interpret recommendations in the context of their experiential knowledge, organizational values, resource constraints, and site-specific conditions (CIEEM, 2021). Guidance that is clear about its underlying evidence will make this easier.
Writing evidence-based guidance includes many challenges. Relaying complex and nuanced scientific and technical information into easy-to-use formats is complicated (Brick et al., 2018;Brick & Freeman, 2019) and there are a variety of ways in which guidance can be presented depending upon the intended audience. Ensuring guidance is accessible to all is essential. Layered guidance is a useful option, that is, a fully referenced technical report can be produced and then presented alongside a more user-friendly version (Cruickshanks, 2018), so that users can select the technical depth appropriate to them. At one extreme, detailed and technical guidance handbooks may give comprehensive references listing each study, while brief advice to the public may simply allude to the existence of unspecified studies (Table 1). With the increasing use of digital media, the space constraints and readability issues caused by including references are no longer as limiting. CIEEM's Principles for Producing Good Guidance also provides suggestions of how to make guidance more user-friendly (CIEEM, 2021).

| DISCUSSION
Our review of UK and Ireland guidance documents for conservation actions revealed many problems, primarily a lack of clear referencing, lack of transparency in how recommendations were made, little updating of guidance, and a lack of disclosing the strength of the evidence. Our principles in Box 1 aim to address these issues by providing clear recommendations on how to produce evidence-based guidance. Ensuring that evidence for the recommended actions is transparently presented will allow users to know where information is sourced. Our principles also suggest a process for how often to review source evidence and update recommendations. Clear documentation of methods and uncertainties leads to transparency in the limitations of the guidance and allows updates to be quicker and easier. Better guidance means that users can more often select better decisions. Our study did not investigate whether guidance documents 8. Make explicit where statements have been made in the absence of effectiveness information. Make cases explicit where no evidence exists and recommendations are based upon first principles, theory or common sense. Consensus recommendations are still valuable when made without scientific evidence, for example, based on practitioner knowledge and experience. Explicitly labeling these cases reveals gaps in evidence-based guidance that inform future research.
9. Make explicit where recommendations are based on factors besides the evidence of effectiveness (e.g., costs, social acceptability). Some recommendations are derived from a range of factors beyond the available evidence base, such as financial costs or the acceptability of outcomes and side effects to different stakeholders. This logic and the key factors should be made clear in the guidance. For example, there may be good evidence for the effectiveness of an action, but it may be too costly or socially unacceptable and so is not recommended.
that contained references to tests of interventions provided recommendations that reflected the results of the referenced paper. This is something that should be explored in further research (Downey, in preparation).
There is an increasing number of resources available to help organizations reduce the time and effort needed to collate and interpret evidence (e.g., conservationevidence. com, environmentalevidence.org, Applied Ecology Resources); however, there are still substantial barriers to making evidence-based guidance. We suggest additional ways that may be more attainable for organizations, including greater co-production of guidance between organizations, building capacity in organizations (Thomas-Walters et al., 2021), funders specifically allocating funds to the production of guidance, establishing unbiased brokers to produce guidance for organizations, and academics providing support and peer-review for guidance production. In addition, having a trusted database that is a repository for guidance could provide an invaluable resource for those seeking advice.
Following the principles set out in this paper means that users can be better assured that guidance recommends effective and appropriate actions and any conservation action that follows the guidance is likely to be more successful. As the concepts described here are generic to any evidence synthesis and recommendation process, the principles could be applied in fields outside conservation where guidance is being produced, or other areas where recommendations should be backed up by scientific evidence.
Conservation practitioners are limited by time, resources, and capacity. Using evidence-based practices successfully largely relies upon the quality and accessibility of evidence summaries and guidance (Sutherland et al., 2004;Walsh et al., 2015). Ensuring the guidance produced is evidence-based and transparent can help practitioners achieve greater conservation effectiveness and could decrease cost.
T A B L E 1 Examples of wording to describe different evidence types in popular accounts when not giving exact details

Evidence base Examples of wording
Evidence assessed and straightforward Evidence shows that ….
Scientific studies indicate that …. The accumulated evidence to date demonstrates ….

Evidence assessed but results mixed
Most evidence supports the idea that …. The evidence is equivocal but overall seems …. The evidence suggests it is likely that ….
Evidence shows no effect The evidence shows no benefit from …. The science shows no support for ….

Evidence assessed but recommendations based on other factors
Evidence shows that … However, due to high financial costs the action is not recommended.
No or weak evidence There are no studies testing ….
There is little research to show …. There are no studies testing … But because of low risk and cost, practitioners should consider …. There are studies but they have flaws or exhibit serious bias that makes findings dubious ….

No evidence but experience used
Although no scientific evidence exists, practitioner experience suggests … Our experience is ….

No evidence provided but uncontroversial
Do not treat shrubs in bird nesting season. Attach boxes by bands rather than nails if the tree is grown for timber.