Bridging the knowledge‐implementation gap between agency and academia: A case study of a graduate research experience

Conservation biology is particularly susceptible to the knowledge‐implementation gap where academic pursuits do not always meet the needs of practitioners. Providing future practitioners with relevant training and experiences as graduate students can help narrow this gap. An example of one such experience was a partnership between the University of California, Davis and The Nature Conservancy (TNC), established to conduct climate vulnerability assessments of wildlife in California. Here, we discuss the value of the experience for graduate students as both developing conservation professionals. Students were motivated to participate because of the opportunity to work on an applied project and valued collaborating with fellow graduate students. Overall, we felt the project successfully met objectives set out by TNC. In recognition of the value projects like these have, we provide a suite of recommendations based on our experience and lessons learned for academic faculty and conservation practitioners wishing to establish similar opportunities.


| INTRODUCTION
Conservation scientists are faced with a difficult challenge: produce novel research results that maintain a high level of academic rigor, while also being timely, relevant and useful to conservation managers (Cook, Mascia, Schwartz, Possingham, & Fuller, 2013). Despite a growing need for applied solutions to pressing environmental issues (e.g., climate change, expanding endangered species list, invasive species), most conservation research does not directly translate into action (Habel et al., 2013;Salafsky, Margoluis, Redford, & Robinson, 2002). The difference in goals between academics and on-the-ground conservation managers has made the field of conservation biology particularly vulnerable to the knowledgeimplementation gap, or "knowing-doing gap" (Arlettaz et al., 2010;Knight et al., 2008;Pfeffer & Sutton, 2000). This gap highlights the need to shift approaches so that scientific findings are relevant to conservation management and policy (Game, Schwartz, & Knight, 2015).
The knowledge-implementation gap in conservation is widened by the unidirectional flow of knowledge from scientists to practitioners and decision-makers (Bertuol-Garcia, Morsello, El-hani, & Pardini, 2018) and a mismatch between objectives of these two groups. Some academics may focus on conceptual approaches to understanding conservation problems and solutions, while practitioners may find those approaches too broad to address issues at a local level (Sunderland, Sunderland-Groves, Shanley, & Campbell, 2009). When there is miscommunication, this gap can lead to substantial ecological and economic costs (Kareksela, Moilanen, Ristaniemi, Välivaara, & Kotiaho, 2017).
Proposed solutions to this issue involve increasing two-way communication and joint knowledge production between scientists and decision-makers (Bertuol-Garcia et al., 2018;Lauber, Stedman, Decker, & Knuth, 2011;Toomey, Knight, & Barlow, 2017), and giving practitioners a role in setting a research agenda (Braunisch, Home,-Pellet, & Arlettaz, 2012;Knight et al., 2008). There are also calls to reward academics for implementation of their research (Knight et al., 2008). These actions have been broadly referred to as boundary spanning (Cook et al., 2013), or translational ecology (Enquist et al., 2017).
Adequate training of graduate students in the particular skills necessary to become successful conservation professionals is one approach that may shorten the knowledgeimplementation gap in conservation science (Knight et al., 2008;Pietri et al., 2013;Schwartz et al., 2017). Graduate students often lack both interdisciplinary training opportunities and institutional support to conduct applied research. Pietri et al. (2013) recommends graduate students work with peers, faculty sponsors, stakeholders, and the general public during their graduate careers to cultivate the skills needed to bridge the knowledge-implementation gap for their future careers. For example, graduate students can benefit from increased preparation in science communication as well as training in the use of tools used by practitioners (Blickley et al., 2012). Participation in interdisciplinary and applied projects can help prepare students to become leaders in conservation science (Muir & Schwartz, 2009), as well as enhance the overall graduate research experience (Courter, 2012).
The University of California, Davis (UCD) has developed a program to provide these types of interdisciplinary training opportunities for graduate students in conservation science. UCD graduate students have the option to earn a Conservation Management Certificate (Box 1) in conjunction with their graduate degree. The program requires students participate as consultants for an external organization (government agency, nonprofit, etc.) on a collaborative conservation-oriented project (e.g., Hameed, Holzer, Doerr, Baty, & Schwartz, 2013). This paper evaluates the utility and efficacy of one such project at bridging the research-implementation gap. The authors of this article comprise most of the team who worked on this project: graduate students (A.K.O., A.K.B., A.C.C., E.E.G., J.A.S., and E.F.T.), faculty member (M.S.W.), and a conservation partner (T.R.K.). To evaluate student experience, we addressed the following two questions: (a) Was the project of value to us as developing professionals? and (b) Did we succeed in producing actionable research for the cooperating partner? We then provide recommendations for academics and practitioners to develop similar collaborative opportunities.

| CASE STUDY PROJECT
In January 2017, UCD graduate students partnered with The Nature Conservancy (TNC) in California to perform vulnerability assessments on a suite of species under different climate change scenarios. The goal of the project was to understand how protecting and restoring habitat could mitigate climate change impacts on 27 species in

Box 1 Conservation Management Certificate
The Conservation Management Certificate is open to graduate students enrolled in any program at the University of California, Davis. The certificate program provides an opportunity to develop marketable skills for students wishing to pursue a career in conservation and management. The certificate requirements include: • A graduate-level course in conservation ecology • A graduate seminar in conservation decisionmaking and planning • Collaboration on a project with a conservation partner to answer mission-directed questions of management relevance Through this certificate program, students gain experience using conservation tools (e.g., Open Standards, Miradi), earn a nominal honorarium for participation in the collaborative project, and receive a transcript notation indicating receipt of the certificate.
three key ecoregions of California: Sierra Nevada forests, rangelands, and the San Joaquin Desert. Students performed climate vulnerability assessments using the traitbased NatureServe Climate Change Vulnerability Index tool (CCVI, V3.02; Young et al., 2011). Graduate students met regularly with one another, as well as with the faculty PI and TNC project lead, to discuss project specifics such as species selection, implementation of climate assessment tools, and ensuring consistent methodology.
The list of species assessed was compiled jointly by TNC and graduate students based on the species' high ecosystem value, endemism, and/or conservation status. Students used publicly available data to compile information for use in the CCVI (Table 1). Each species was evaluated under two possible climate model scenarios (CNRM-CM5 and MIROC-ESM), which have been used in previous climate vulnerability reports in California (Thorne, Boynton, Holguin, Stewart, & Bjorkman, 2016). The CCVI tool generated vulnerability scores and confidence levels in those scores. Details regarding vulnerability assessments and scores are summarized in Box 2. For complete details on methodology and results see the full report produced by Graves et al. (2017) (see Supporting Information).

| Assessing project value
In order to understand the project's effectiveness at bridging the knowledge-implementation gap, we used post-hoc survey questions to evaluate how the views we (i.e., graduate student participants and TNC project lead, TRK) held before the project changed upon project T A B L E 1 Species information and method of collection used in the climate vulnerability assessment tool, NatureServe CCVI (Young et al., 2011)

Box 2 Climate vulnerability assessments of California wildlife
We conducted climate vulnerability assessments of 27 species across three California ecoregions ( Figure B2.1): Sierra Nevada Forest (n = 13); San Joaquin desert (n = 8); California rangelands (n = 6). Species were selected based on their high ecosystem value, endemism, and/or conservation status. We used two downscaled Global Climate Models that are frequently used to represent the bookends of potential future climate: the CNRM-CM5 predicts a warmer and wetter future and the MIROC-ESM predicts a much warmer and drier future. We found that our 13 focal taxa from the Sierra Nevada Forest region range predominantly from less vulnerable to moderately vulnerable with only two species (Sierra Nevada yellowlegged frog, Sierra Nevada red fox) scored as extremely vulnerable to climate change under the more extreme MIROC climate model (Table B2.1). In contrast, all eight focal species (five mammals, two birds, and one reptile) in the San Joaquin Desert scored as highly or extremely vulnerable under both climate models (Table B2.1), as this ecosystem is projected to become much warmer and much drier under both future climate models. Rangeland species (three mammals, two birds, and one amphibian) ranged across the spectrum from less vulnerable under the more moderate future climate projection (CNRM, burrowing owl, bobcat) to extremely vulnerable under either climate model (California tiger salamander).
Our findings suggest that climate change exacerbates current conservation challenges where species with on-going habitat loss (e.g., California tiger salamander, San Joaquin kit fox), disease and predation (e.g., Sierra Nevada yellow-legged frog) appear considerably more at risk to changing climate than those species that are of concern, but appear to be stable on the current landscape (e.g., bobcat, willow flycatcher). Some species that are declining as a consequence of other stressors (e.g., Pacific fisher, California spotted owl) do not appear inordinately vulnerable to changing climates. completion. We used survey questions to structure our conversation about the project's value and to guide reflection on how our perceptions changed (see Supporting Information). The goals of these survey questions were to (a) understand the value of collaborative conservation projects to graduate students and the partnering organization and (b) determine how these projects help foster skills necessary for careers in conservation. The survey questions simply provided a means for our group to collate our collective opinions. As such, we do not test hypotheses or use analytic statistics. Instead, we describe the results as a means of summarizing our perspective.
To understand our value of this project, we asked ourselves what motivated our participation and what we found the most rewarding after project completion. We developed a list of eight possible motivations for participation (Table 2). Each student ranked those motivations and noted any additional motivations they had. Comparing initial motivations to the perceived benefits gained after participation provides a metric of how expectations measured up to outcomes and how our sense of the project's value changed through participation. However, all questions were asked after project completion, therefore, surveyed interest in the project only presents a reflection after the fact. Additional survey questions addressed skills useful for future careers as conservation professionals such as using conservation tools (i.e., NatureServe CCVI) and presenting research to practitioners. To understand the value of this project to the partnering organization, the TNC project lead (T.R.K.) was asked about project expectations and outcomes, and the effectiveness of partnering with graduate students.

| Graduate student value of project
Upon reflection, the most highly ranked motivations for participation were the opportunity to work with TNC and on an applied project. Overall, we ranked our motivations for participating in the following order from most to least important: (a) working on an applied project, (b) working with TNC, (c) learning to use conservation tools, (d) completing CVAs, (e) opportunity for publication, (f) monetary incentive, (g) working with other graduate students, and (h) completing a course requirement.
While most rankings changed after participation, working on an applied project remained the most rewarding aspect of the project (Figure 1). Completing CVAs and collaborating with other graduate students both increased-the latter notably from seventh to third most important-leading to slight decreases in importance of the remaining aspects. Fulfilling a course requirement remained least important.
One specific value many of us gained through this project was a greater understanding of how species' data are used and an understanding that uncertainty may be perceived differently between graduate students and decision-makers. Specifically, we had high confidence in Graduate students received a $500 honorarium for their work. c Potential for publication of work discussed at onset of the project.
F I G U R E 1 Change in overall ranking of aspects of interest before and of value after completion of a collaborative climate vulnerability assessment project with The Nature Conservancy the use of NatureServe's CCVI tool (mean rank of 7.3 out of 10) because it incorporates a broad range of variables from the literature and allowed input regarding uncertainty. However, we had lower average confidence in the effectiveness of the tool than TNC project lead (T.R.K.), who ranked confidence at 8 out of 10. Students had less confidence specifically when the species of interest was not well-studied and because of the subjectivity of translating published information into a vulnerability score (Figure 2). The higher confidence of T.R.K. compared to F I G U R E 2 Quotes from graduate students surveyed following a collaborative climate vulnerability assessment project with The Nature Conservancy graduate students may be a product of comfort level with making decisions under uncertainty, which is frequently necessary when taking conservation actions.

| Project impact
When asked to rank how useful the project was in meeting the objectives set out by TNC (0 = useless and 10 = invaluable), the average student response was 7 out of 10. Some of us noted that though we were confident in our work, we did not have enough direct correspondence with TNC to accurately assess how useful the results were for the organization. On average, our understanding of how research informs conservation increased, from 6 (out of 10) before the project to 8 after the project. This project provided us with insight into TNC's goals and decisionmaking processes, and for some of us, a better understanding of how their own research may be used by external agencies. Additionally, the project provided concrete skills to effectively translate academic research into a form useable by decision-makers, including synthesizing research for climate vulnerability assessments and writing findings as a report rather than an academic journal article.
Upon project completion, we did not have high confidence (average 6 out of 10) in our ability to communicate research to managers. The results were ultimately not presented in-person to TNC staff and there was limited time for follow-up about how the results would be used. As with many projects, this work took longer than expected to complete, and the primary TNC contact, although interested in the project, was focused on other priorities at the time of completion. Improving project closure is one of the areas of broad agreement we could have done better.
When asked about the impact of the project for TNC, T.R.K. said that the product of the project exceeded expectations and that the graduate students involved were able to take the analysis further than expected, resulting in a thorough, polished report that may not have been possible otherwise. The climate vulnerability report will ultimately be incorporated into a statewide assessment of lands and waters protection priorities for TNC based on protecting biodiversity and connectivity. T.R.K. stated the least effective aspects of the partnership had to do with time constraints that limited collaboration and communication between TNC and graduate students, a sentiment shared by the students.

| RECOMMENDATIONS
Based on our experience (Table 3) and lessons learned (Table 4), we provide the following recommendations for academic researchers and conservation practitioners to develop similar collaborative projects.

| Formalize partnerships
We recommend creating a formal contract between the academic institution and partnering organization. If the partnership is established for a single project, the contract can include an expected timeframe for completion, clear deliverables, and anticipated compensation for the students. A long-term partnership may require a contract outlining the frequency of projects and an overall budget.

| Project selection
When selecting a project, it is important to consider issues of feasibility and scope. A project must be timely for the partnering organization and feasible within the F I G U R E B 2 . 1 Geographic extent of the three focal ecoregions in California: San Joaquin Desert, Rangelands, and Sierra Nevada Forest available timeframe. The timeframe of the project should be selected to maximize student involvement; a single academic term is often convenient for student groups because of other graduate responsibilities. Due to the short timeframe and to avoid delays often associated with collecting data, we suggest selecting a project for which data has already been collected or which relies on publicly available data. It is also important to plan and discuss how the workload will be distributed amongst students. Finally, continual dialogue on how the end product will be used can inform students on how their work is helping to bridge the research-implementation gap.

| Recruiting students
First, we recommend recruiting graduate students by structuring project participation within a course which can serve as an effective way to attract students and create accountability. Courses that teach tangible skills while fulfilling coursework requirements provide valuable experience without taking time needed for other academic obligations. Second, we recommend recruiting from a broad suite of disciplines, possibly through other courses. Generalized courses, such as conservation science, are likely to draw students from many T A B L E 3 Recommendations based on our experience for academics and practitioners wanting to establish similar collaborative projects

Recommendations
Our project

Formalize partnerships
Organizational contracts (funding, deliverables, timeline) A UCD-TNC contract was in place that specified funding, deliverables and a timeline.
Student contracts (compensation, deliverables, expectations, timeframe) Student contracts were verbal, compensation and deliverable were explicit, timeframe and commitments were less clear.

Project selection
Agency driven (actionable, compelling need) Project was clearly linked to TNC management objectives; although less clearly linked to future action.
Timeline (<1 academic year) Project was timely and was defined with a good timeline.
• Most work completed within academic quarter • Report and manuscript finalized afterwards Feasibility (e.g., no new data collection) Project was highly feasible; used publicly available data Scope (group size to workload, required skillsets) Group size was appropriate and contained necessary skillset • Created region-specific teams where each student assessed a few species

Recruiting students
Class credit Project was offered for class credit and counted toward valued Conservation Management Degree Certificate Skill sets Skillset diversity was unplanned, but ultimately fruitful • Students had prior expertise in some of the assessed species and/or ecoregions Advertising Projected benefits of participation not clearly presented

Identify leadership
Partner organization contact Partner organization liaison struggled with time commitments Academic coordinator Academic leadership was satisfactory Graduate student coordinator Student leadership was outstanding

Outline clear expectations
Continually revisit deliverables Communication was excellent among students, but less than ideal with partner organization Time commitment and scheduling Partner organization remained flexible regarding outputs

Assess project effectiveness
As a learning opportunity Academic leadership neglected to launch pre-project survey of student expectations. Post-project assessment was successful.

As a deliverable
Partner organization provided feedback on use of deliverable.
In-person reporting to the partner by the students was delayed, then canceled.
sub-disciplines, which ultimately leads to fruitful collaborations among graduate students. Since students were highly motivated by the promise of doing applied work, recruitment should highlight this feature of the course.

| Identify leadership
It is helpful to clearly establish leadership for the project, including the academic coordinator(s) (typically a faculty member), the agency contact(s), and the graduate student coordinator(s). The professor(s) should oversee the academic logistics of the project (e.g., facilitating course or research requirements). They may also develop (or draw on existing) partnerships with agencies. The agency contact (employee or manager) should guide the development of meaningful research goals. This person(s) should communicate with the academic coordinator and graduate students throughout the project. We also recommend designating a graduate student to consolidate communications between the academic coordinator, agency contact, and graduate student research team. This student (or another student) could also be responsible for coordinating the final report or similar deliverables, especially if they extend beyond the timeframe of the course. Clearly identifying roles and responsibilities for each team member is crucial for transparency and accountability and promotes effective communication between different contributors.

| Outline clear expectations
We recommend that graduate students and project partners clearly define the project scope at the onset. This should include identification of participant expectations, availability of resources (software, finances, time, etc.), specific tasks, and clear deadlines. To improve the project's impact, defining the scope should include explicit consideration of the audience (i.e., any end users beyond the agency partner), the scope and type of information needed given the problem being addressed, and how to best communicate the results once done (Fisher, Wood, Bradford, & Kelsey, 2020). Establishment of a contract, Memorandum of Understanding (MOU) or similar formal document can recognize expected deliverables. Additionally, expectations and directions are likely to shift as projects develop, so within the scope, allow for flexibility in project direction. Schedule periodic meetings and document these changes. Once the project is complete, review the project scope once again to identify any outstanding deliverables, reflect on accomplishments,

| Assess project effectiveness
We recommend that organizations implementing similar projects conduct surveys of participants (students, professors, and agency) before, during, and after the project. Surveys at the start of the project would provide transparency about the expectations for, and feasibility of, the project. Identifying individual motivations and expectations allows for the opportunity to adjust the project at the start to provide appropriate experiences. Surveys conducted during the project provide checkpoints to ensure that everyone is adhering to the contract, producing the desired deliverable, and gaining the desired experience while also highlighting unexpected challenges. Lastly, surveys conducted upon completion of the project may illuminate areas for improvement for these types of collaborative projects and allow recommendations to be built upon what we outline here. This feedback can be particularly important in the early stages of development of long-term programs such as the UCD Conservation Management Certificate Program.

| CONCLUSIONS
Given that more graduate students are pursuing career paths outside of academia (Roach & Sauermann, 2017;Sauermann & Roach, 2012), it is important that graduate programs provide training in how to translate conservation research into practice. Traditional graduate programs focus on skills that are valuable in academia such as conducting novel research, publishing in peerreviewed journals, and collaborating with and presenting to other academic scientists. Although some of these skills are translatable to non-academic careers, opportunities to develop other desirable skills such as synthesizing literature to inform policy and management decisions, writing reports and white papers, familiarity with tools used by NGOs and government agencies, science communication, and collaborating with practitioners and other non-academic scientists are not common in graduate school. Providing project-based courses in collaboration with external organizations is an effective way to introduce graduate students to these skills. Although a single experience cannot be expected to provide all of the skills necessary for a student to adeptly bridge the research-implementation gap, providing such opportunities can increase confidence and encourage graduate students to pursue similar collaborative projects in the future. Though this approach has limitations and should not be viewed as a panacea, it is an important first step towards actively narrowing the research-implementation gap.

ACKNOWLEDGMENTS
We thank Karyn O'Hearn and Arturo Ramirez for their work on the climate vulnerability assessments and responding to the questions posed for this manuscript. We thank Hyeyeong Choe and James H. Thorne for their consultation on climate models, generation of climate projections and vulnerability of vegetation in California. Finally, we thank the Nature Conservancy for providing the funding for the climate vulnerability assessments project.