Evaluating the contribution of a wildlife health capacity building program on orangutan conservation

One Health is increasingly being used as a tool in ecosystem protection. The Orangutan Veterinary Advisory Group (OVAG) is working to address One Health concerns in Pongo spp. (orangutan) welfare and conservation. Orangutans are vital contributors to the ecosystem health of their range areas. Strengthening national capacity is crucial to make a lasting difference in the currently bleak outlook for orangutan species survival. OVAG is a capacity strengthening and expertise network that brings together all those working with orangutans, in‐ and ex‐situ, to share knowledge, skills, and to collectively learn. Using the One Health paradigm embedded to enhance professional development, the OVAG network is successfully supporting conservation outcomes and impact. As part of our adaptive management approach, and to assess individual and organizational change attributable to the capacity strengthening work of OVAG, we evaluated technical skill test data, program satisfaction data, and asked participants to complete a self‐reflective survey. This pilot study of our work demonstrates statistically significant improvements in conservation medicine (t = 5.481, p < 0.0001) and wildlife clinical skills knowledge (t = 3.923, p < 0.001) for those in the OVAG program. Most consider OVAG participation to be either critical or very useful in their conservation medicine decision‐making process, with a perceived positive impact on their skills at handling multiple situations. Additionally, participant feedback shows a sense of being able to drive positive change locally and nationally (within their own countries) as a consequence of OVAG participation. The authors hope the OVAG model including its associated capacity support mechanisms and pedagogical approaches can be used as a template for other One Health efforts.

. Recent reviews on the health of orangutan and other ape populations indicates habitat loss and fragmentation puts undue stress as access to food is reduced, mating opportunities decrease, and vulnerability to disease increases (Calvignac-Spencer et al., 2012;Kondgen et al. 2008;Utami-Atmoko et al., 2017). This is exacerbated by increasing population densities in remaining habitat which may trigger conflict between orangutans, increasing mortality, inbreeding, and facilitating disease outbreaks.
Disease may also pose a risk at the individual population level where populations are small and isolated leading to genetic diversity loss.
Where there is overcrowding (e.g., after fires), or regular disturbance from human-mediated activity on orangutan behavior (e.g., tourism, orangutan reintroduction/translocation), there may be an increased exposure to potential disease agents (Ditjen KSDAE, 2019, Dunay et al., 2018, Kilbourn et al., 2003, Mul et al., 2007. The pace of new disease outbreaks has led to increasing recognition that emerging infectious diseases originate at the interface of human and animal ecosystems (Woldehanna & Zimicki, 2015). This recognition underscores the need for an inter-disciplinary approach in dealing with disease transmission in already threatened ape species (Estrada et al., 2017). The One Health paradigm continues to evolve (Gruetzmacher et al., 2020), but fundamentally integrates investigations of wildlife, livestock, crop, and human health and welfare in an ecosystemic context (Wallace et al., 2015). The approach convenes medical doctors, veterinarians, ecologists, and sociologists under the rubric that many species share infectious, chronic, and environmentally sourced illnesses.
Data on emerging diseases demonstrate the overlap between locations of emerging zoonotic diseases and biodiversity hotspots (Jones et al., 2008). This suggests that One Health practices go beyond human and animal biological surveillance and incorporate the human social and structural contexts where emerging zoonoses are most likely to occur, such as Indonesia and Malaysia (von Rintelen et al., 2017), the only two countries where orangutans are indigenous.
Due to its tropical climate, vast forested areas, and high human population, Indonesia is a global hotspot for zoonotic disease emergence from animals, especially wildlife (Wilks & Fenwick, 2009).
With 60% of zoonotic diseases coming from wildlife, the role of veterinarians in wildlife medicine is imperative for disease surveillance.
Launched in 2014, the Global Health Security Agenda (GHSA) brings together national governments, the WHO (World Health Organization), OIE (World Organization for Animal Health) and others, to provide a global mechanism for One Health collaboration and to enhance disease surveillance. GHSA identifies a critical lack of adequately trained public and wildlife health professionals (Wolicki et al., 2016, GlobalHealthSecurityAgenda,2016. From personal observations and discussions with various veterinary faculty in Indonesia, wildlife medicine and orangutan health and welfare specifically, has not been a focus of Indonesian veterinary schools. Calls to improve One Health tertiary education or community and clinical practitioner capacity have been made from the human medical field in different settings (Bolon et al., 2020;Rweyemamu et al., 2012), but there remains a need to identify effective pedagogical approaches.

| Introducing the OVAG program
The Orangutan Veterinary Advisory Group (OVAG, www.ovag.org) was established in Indonesia in 2009, as a technical capacity building and networking initiative. Most young veterinary clinicians working in orangutan range countries have limited wildlife training within the university system and have had to "learn on the job" often in isolation from external veterinary assistance. OVAG identified that investing in a capacity strengthening program for key individuals in orangutan health management would provide the much-needed support and platform for collaboration, increasing the impact of these individuals toward more successful efforts to effect orangutan welfare and conservation. The development of the OVAG community was designed to fill the veterinary led tier of the GHSA highlighted need for capacity development and advocacy for the inclusion of One Health principles in orangutan action plans. OVAG brings together those working with orangutans to share expertise, knowledge, and collectively learn to increase effectiveness of orangutan welfare and conservation. This community includes veterinarians, academics, researchers, project managers, and government authorities from Indonesia, Malaysia, and around the globe.

| Designing a capacity building program
Capacity development, as with any initiative, requires management principles to help build a strong foundation. Following Kern (2016) OVAG developed a six-step approach for wildlife veterinary education in Indonesia and Malaysia: 1. Identify the problem: In both Indonesia and Malaysia, there are a lack of science graduates in general versed in One Health to combat emerging and pandemic disease.
2. Targeted needs assessment: Define minimum competencies for specific disciplines, which in this case is wildlife health and welfare.
3. Setting goals and objectives: Developing a plan (outlined as OVAG Theory of Change, Commitante et al., 2017). 4. Developing an appropriate educational strategy: Holding annual workshops with a combination of didactic lectures, focus discussion groups, and hands-on trainings, bringing together multidisciplinary experts across university, medical and zoo veterinary networks, encouraging collaboration and increased hands-on experience and development of critical thinking skills through problem-based learning.
5. Curriculum review and design: Using ongoing participant evaluation and feedback, and annual review, keeping with the network's strategies.
6. Implementation, evaluation and feedback: Ongoing participant evaluation, online survey, annual workshops, and other outputs to ensure that findings from the feedback are integrated into the ongoing design of projects and initiatives.
The underlying assumption of evaluation is that better information on what does or does not work will drive better decisionmaking and in turn better conservation (Stem et al., 2005). This article presents a pilot evaluation of the current efficacy of OVAG activities on its participants where data were able to be collected (2015, 2016, 2017, and 2019). It presents a mixed evaluation approach that could be of use to the medical, One Health and conservation fields alike. As a pilot, these represent first steps to confirm OVAG is utilizing good pedagogical practice to effectively allow participants to learn, acquire, and implement knowledge gained. The next stage will be to include more quantitative measures for skill evaluation and confirm that program outputs also integrate modern One Health practice as per the Berlin principles (Gruetzmacher et al., 2020).

| Pedagogical justification and evaluation limitations
A good learning environment engenders acceptance and a sense of empowerment via community-building through problem or scenariobased learning (Broom, 2015). This was especially relevant when creating OVAG which is a Continuing Professional Development (CPD) program for practitioners in a multicultural learning situation, where social etiquette and ways of understanding greatly varied among participants. An interdisciplinary, multinational approach also lends itself to the One Health approach from an educational point of view. Bowden et al. (2019) provide recent support for our pedagogical approach as OVAG has evolved by discussing the four pillars of engagement-social, affective, behavioral, and cognitive-to manage expectations and outcomes and to guarantee program involvement. Like any tertiary academic teaching module, some OVAG participants change each year, but unlike a university course, others dip in-dip out, or have been there throughout. This produces audiences with different levels of capacity need. As such, evaluating the program must take the semi-quantitative approach as utilized. The underlying assumption of our pilot study is that better information on what does or does not work will drive better decision-making (Stem et al., 2005). Thus, we have focused on knowledge gained in conjunction with participant evaluation to semi-quantitatively assess the value of the program to One Health practice, in line with recognized best practice pedagogy. OVAG attempts to follow the aligned teaching model as described by Biggs (2003) where curriculum content and outcomes, teaching methods, and assessment tasks are aligned to each other, to guarantee these different audiences are catered for. This approach is further aligned with the orangutan veterinary sector's needs, via participant feedback, for producing interdisciplinary competent professionals (Amuguni et al., 2017).
Understanding which actions are effective or not and why, avoids wasted effort and funds, enables resources and funding to be prioritized, improves delivery, and provides accountability for participant employers, governments, and donors. To confirm success in learning outcomes and that participant support was being properly targeted, input was gathered over several sessions.

| METHODS
All work conducted at the annual OVAG workshops was with an in-country partner (Gadjah Mada University) and was further

| Part II: Participant feedback
A feedback form, based on Return on Investment (ROI) principles, was developed by the Discovery and Learning team at Chester Zoo, UK. This resulted in a semi-quantitative assessment using a five-level scale from strongly agree to strongly disagree, given to participants at the end of each workshop from 2014 to 2019. Participants were asked if they gained new knowledge or ideas, if they would use information gained, if they had been shown how to impart knowledge to colleagues, and finally if ideas and information provided at the annual OVAG workshops would improve the way they do their job and the health of the animals under their care. The second part of the form asked for comments on how to improve the workshops, and how they will use the information they have gained. Both parts of the form were in English and Bahasa Indonesia and participants responded in the language they felt most comfortable. An example of the feedback form is in Supporting Information Appendix 2.

| Statistical analysis
The results of quiz answers were consolidated into "conservation medicine" and "clinical skills" aspects and analyzed using independent t tests to assess differences in knowledge between preand posttests. Tests were marked by the experts that wrote them and were validated by independent veterinary expertise at Chester Zoo, UK. An overall score was also assessed using the same methodology.

| Methodology limitations
The purpose of this pilot evaluation is to confirm or deny that our pedagogical approach to capacity development is perceived by participants to be what is needed. As such, the OVAG network incorporated all opinions within the survey analysis, as there was no "conflict of interest" or bias as all opinions were considered valid. The quizzes were administered only to participants (and not facilitators).
Some members of the OVAG committee, who did not devise or had access to any quiz before testing, but who work in orangutan centers and/or academia in orangutan range countries were categorized as participants. Therefore, most of the data presented here are based on self-reported feedback. The opportunity for participants to provide both positive and negative feedback anonymously was provided in the annual feedback form.
Data for points number two and three from the list above is collected semi-quantitatively in the survey to measure the effectiveness of OVAG's capacity building programs. Options are provided on a continuum scale, ranging from (capacity) increased greatly to (capacity) decreased greatly. Further, we asked an open question on any contribution that the participants have made and how much OVAG affected those contributions. Data for point number four were also collected semi-quantitatively by providing scalable options ranging from not at all (as a barrier) to complete barrier of internal and external factors that may challenge participants in giving contributions to conservation and One Health.

| RESULTS
Results are presented according to the corresponding evaluation approaches as per the previous section.

| Part I: Pre-and postworkshop knowledge gain as a group
Because responses were anonymous to encourage participation, pre and postparticipation responses could not be matched to individuals (see Table 1), so data were analyzed using independent t tests.
Data showed a statistically significant improvement in knowledge from before to after tuition and participation ( Figure 1). This improvement was seen in both areas under examination: general conservation medicine management (t = 5.481, p < 0.001) and wildlife clinical skills (t = 3.923, p < 0.001) for a total of t = 5.053, p < 0.001. This was in both general conservation medicine management knowledge, (t = 7.56, p < 0.001), and wildlife clinical skills (t = 4.01, p < 0.001), for a total of t = 6.9, p < 0.001.   (Table 2), and this sentiment remains constant over the time period. Just over 20% also identified that participating in the workshops gave them a sense of community and empowerment. Improvements were predominately focused (43.2%) on suggestions for future topics and sessions. Just over a third of participants (34.6%) highlighted the need for more hands-on training sessions on various technical skills. Over half of participants (54.4%) intended to share information both internally (with management and colleagues) and externally (with government and students), and just under half (42.6%) intended to use the knowledge and skills learned to improve how they do their job (this was often presented jointly with sharing knowledge).

| Part III: Participant online survey
The online survey was completed by 55 persons, with slightly more females than males ( OVAG is attempting to develop "Pathways to change" for the network's mission through providing capacity building and orangutan health research and education opportunities, and by being actively engaged in policy and advocacy in relation to orangutans and their environment. Qualitative evidence of impact into this has been highlighted in the methods section. This process follows the principle of "build the movement from the bottom, strengthen it from the middle and top" (Reid et al., 2002). Veterinarians are often structurally positioned under the operational manager, who then reports to the director of the organization.
Building the capacity of the veterinarians follows the hope that they would act as catalysts for scientific-based practice in their centers as well as encourage more collaboration at the organization level.

| Improvement in transferable and technical skills
The OVAG "model" is a multidisciplinary network that brings capacity development, research, policy, and advocacy together to improve orangutan health, welfare, and conservation. This article has focused on the beginnings of formally evaluating the impact of capacity development activities of the network on participants.

| Participant evaluation and feedback
Continuous feedback from participants of capacity building activities is needed to allow confirmation (or not) that the program is meeting participant needs. Continuous performance evaluation is needed to confirm effective transfer of knowledge and skills. This evaluation process is acknowledged to be time demanding and challenging (Britton et al., 2008). As an iterative process, and in line with the OVAG Theory of Change (Supporting Information Appendix 3), we will be continuing the evaluation annually with a follow-up survey in 2021, to help guide our strategy for the following 5 years.
The mandatory annual feedback form for all participants to fill in anonymously suggested that the most important trait of the annual workshops they most appreciated was the open environment where ideas and knowledge were exchanged. This indicates that all participants feel equally respected and that everyone has something to learn from one another. We acknowledge that the "experts" in orangutan conservation and health management are the local veterinarians who are working full-time in all 10 rehabilitation centers in Indonesia and Malaysia.
Just over 20% of participants experience a feeling of community and empowerment, which is in part helping lead to a reduction in the high turn-over of veterinarians working in the field. Eleven years after the beginning of our program, 50% of the original participants are still involved. Although many remain in orangutan health, some have moved away from being field veterinarians while some have been promoted or moved into decision-making positions. All still attend annual workshops at their own expense, highly suggestive that the OVAG community is a positive experience for professional and personal development.
After each workshop, the number of participants who asked for more practical sessions was significant (34.6%). This is consistent with topics requested for future workshops, which are heavy on technical veterinary skills such as radiography, dental examinations, ophthalmology, and surgeries (orthopedy, soft tissue, etc). This may reflect the lack of opportunities for hands-on training in a veterinary residency program. Indeed, such programs are rare to nonexistent in Indonesia, especially so for zoo and wildlife veterinarians. We have identified this gap and have helped with getting participants into training programs abroad. We actively look for collaborations to increase such opportunities for Indonesian and Malaysian wildlife veterinarians.
There was a strong will from the participants to share the knowledge and information they received from the workshops to their colleagues (54.4%), as well as using workshop materials to improve their daily practices (42.6%). Obviously, it is very challenging to measure how much OVAG has contributed to the improvement of veterinary and rehabilitation/husbandry practices in orangutan centers, however, it was confirmed from the online survey issued in 2017 that over 75% of respondents think that their participation has been either critical or very helpful to their contribution for the conservation of orangutans in general (Figure 4).

| Online survey results
Interestingly, personal development was highlighted by participants as the greatest contribution by OVAG. The personal development stemmed from networking and being able to build relationships at the annual workshops not just internationally but even between centers. This reinforces the "conservation is people" concept, one of the founding principles of OVAG. Whether this adds to a person's ability to improve the welfare and conservation of the species remains to be concretely proven, but they are more likely to be able to take control of any given situation. An example would be orangutan nutrition. Typically, center veterinarians are not trained in orangutan nutrition, but often they are the ones required to manage it. Nutrition was a main topic at several workshops (2011 and 2019), and as a result, many centers report that they have an improved nutrition plan for their orangutans. Orangutan centers such as the Sumatran

Orangutan Conservation Program, Frankfurt Zoological Society
Sumatra, and the Borneo Orangutan Survival Foundation all have nutrition programs that were initiated or refined by the veterinarians empowered by the knowledge they received from the workshops.
While there needs to be continuous evaluation on nutrition, these vets have put into practice what they have learned to improve the work done by their home organizations. An important outcome of any capacity development is to increase participants belief that they can effect change (Berrian et al., 2018;Copsey, 2017;Sarti et al., 2017). This nutrition example illustrates the focus of empowering participants and creating ownership of the issues and work undertaken so that it remains relevant to them although we acknowledge that there is no data on the direct impact of the improve nutrition plan on orangutan health and welfare. The results presented here highlight their sense of ownership and that the OVAG community is strong. Participation and learning play heavily in their conservation medicine decision-making processes, increasing their ability to provide guidance.

| Barriers to success and pathways to resilience
Didactic lectures still form a large component of the OVAG work- shops. Discussion times are often seen as too short and can easily lead to participant frustration at not being heard, especially in larger groups. The language barrier and cultural differences are also perceived to be challenges in delivering a comprehensive message in the topics presented, which is mirrored in the feedback form. The language barrier has always been mentioned by participants for future improvement. We are currently investigating purchasing handheld translation devices for the next workshop as a possible solution.
Every year, we receive a constantly increasing number of interested people who want to join. Although this could be reflective of the programs respect within the practitioner community, we must acknowledge that larger groups may also be a limiting factor to de- It is widely accepted that successful conservation work is dependent on people (Claus et al., 2010, Scarlett & McKinney, 2016. As such any conservation capacity building program should support the mental health and well-being of its participants. An increasing area of concern in the animal health profession are issues of burn out and compassion fatigue (Hill et al., 2019;Lloyd & Campion, 2017;Volk et al., 2018). The participant feedback results indicate issues of mental health are a concern and the authors will be following this up with a targeted program to assess mental well-being which we will report on in a future paper.

| Pedagogical approach
The principal goal of teaching interdisciplinary approaches such as One Health is to cultivate scientific inquiry that requires coordination of both knowledge and skills simultaneously. By promoting inquiry and discovery in applied real world problems we can enhance the participants' abilities to construct their own knowledge through the relevant hands-on and minds-on activities (Osman et al., 2013;Owen, 2015). An interdisciplinary approach also encourages a wide  c) An effective peer support network at the heart of this One Health program.
To confirm success in learning outcomes and that participant support was being properly targeted, time was needed to gather data over several sessions, to confirm consistency. OVAG is also focusing increasingly on evaluation, especially as we begin the process of Theory of Change review, as we need to be able to quantify indicators and confirm assumptions in line with activities and goals.
The critical characteristics of instruments in educational measurement are validity and reliability (Hecker & Violato, 2009). Multivariate analyses and procedures for investigating levels of analysis of validity need to be explored by OVAG as we continue to investigate impact.
Regarding reliability, these same approaches that involve retesting after some lapsed time (i.e., test-retest, parallel forms at different times) also estimate teaching reliability. As veterinary educators we need to improve our understanding of each of these concepts going forward because evaluation and performance measures provides evidence of the utility of assessment tools, informs future pedagogical practices, and will further guide OVAG's curriculum development and revision going forward.

| Welfare and conservation impact
We understand the link between welfare and conservation and the OVAG community has indirectly improved the welfare of the animals and the conservation focused decision process within its network organizations. However, directly linking OVAG's capacity development with long-term welfare and conservation impact remains elusive. Assessing the impact of orangutan rehabilitation and release on conservation outcomes has only recently started to be externally investigated and evaluated (Sherman et al., 2020).
Although OVAG's theory of change speaks to creating a "sustainable cadre of professionals" to provide this impact, the sector needs to improve its evaluation processes to confirm efforts lead to impact over the coming years. OVAG has run sessions on ethics, animal welfare and behavior, and encouraging participants to share their cases/problems related to behavioral issues. In 2014, a welfare statement we produced was adopted by our network organizations that care for wildlife (Commitante et al., 2015).
However, in the absence of robust welfare assessment (an issue faced by all captive ape institutions) data do not exist to be able to quantify this assumption. internationally. This committee is now leading the scientific program and all workshop logistics (Commitante et al., 2015(Commitante et al., , 2017. These committee members have shown a natural aptitude for understanding the bigger picture of conservation and environmental management in their own countries, and how animal health is a component of this. They act as ambassadors and advocates in their respective areas on behalf of the network. This ethos recognizes the social dimension of conservation and welfare areas of One Health.

| CONCLUSION
While we cannot provide evidence of the direct impact of the OVAG program on orangutan conservation, we can state that: orangutan range country members of OVAG no longer work in isolation, they are in constant contact with a support system that contains over 250 individuals, they can share successes and failures; they can get and give advice; they are building standardized protocols for care; they have been able to access international internship/work exchange opportunities to further their knowledge as well as share their knowledge with others; they have presented at conferences and coauthored published papers; and they have gained a seat at the table in government meetings regarding orangutan conservation (Commitante et al., 2009(Commitante et al., -2012. A Theory of Change has guided the current and future strategy and impacts (Supporting Information Appendix 3). This current research represents a pilot evaluation based on OVAG participant feedback trends to help validate both pedagogical approach and participant needs as we build better efficacy of One Health communities.

ACKNOWLEDGEMENTS
The authors would like to thank all OVAG participants, both individuals and institutions, who took part in providing data for this paper. They are the giants on whose shoulders we are calling their achievements out from. Publication of this paper provides evidence of the OVAG program impact and might assist in gaining increased funding from OVAG's current funders or potential new funders.

ETHICS STATEMENT
Informed consent was verbally obtained from all individual participants that chose to participate at the time of data collection. All data was anonymised for analysis and no identifying data of participants was used in the study. There are no animal subjects in this article.

DATA AVAILABILITY STATEMENT
Data available on request from the authors.