Stakeholder perspectives on the implementation and impact of Indigenous health interventions: A systematic review of qualitative studies

Abstract Background Evaluations of health interventions for Indigenous peoples rarely report outcomes that reflect participant and community perspectives of their experiences. Inclusion of such data may provide a fuller picture of the impact of health programmes and improve the usefulness of evaluation assessments. Aim To describe stakeholder perspectives and experiences of the implementation and impact of Indigenous health programmes. Methods We conducted a systematic review of qualitative studies evaluating complex health interventions designed for Indigenous communities in high‐income countries. We searched 6 electronic databases (through to January 2020): MEDLINE, PreMEDLINE, Embase, PsycINFO, EconLit and CINAHL and hand‐searched reference lists of relevant articles. Results From 28 studies involving 677 stakeholders (mostly clinical staff and participants), six main themes were identified: enabling engagement, regaining control of health, improving social health and belonging, preserving community and culture, cultivating hope for a better life, and threats to long‐term programme viability. Conclusion The prominence of social, emotional and spiritual well‐being as important aspects of the health journey for participants in this review highlights the need to reframe evaluations of health programmes implemented in Indigenous communities away from assessments that focus on commonly used biomedical measures. Evaluators, in consultation with the community, should consistently assess the capacity of health professionals to meet community needs and expectations throughout the life of the programme. Evaluations that include qualitative data on participant and community‐level outcomes can improve decision‐makers' understanding of the impact that health programmes have on communities. Patient or public contribution This paper is a review of evaluation studies and did not involve patients or the public.


| BACKG ROU N D
Evaluations of health programmes designed for Indigenous people are not always considered useful when making decisions about the development or future of health programmes, constituting a significant waste in resources. 1-3 The focus on Western biomedical measures often means evaluations exclude outcomes that reflect participant experiences providing an incomplete picture of processes that are working well and those that are failing. 1,4 Increasingly, evaluations are using qualitative methods to capture participant experiences; however, this is often an afterthought or to support quantitative evaluation data and little attention is given to the importance of qualitative data. As qualitative evaluation data on the experiences of Indigenous people with health services grows, it is clear that these data should be given prominence and should be routinely collected and reported in evaluations of health programmes to ensure decision making reflects the needs of Indigenous communities. 1 A fundamental challenge when conducting a programme evaluation is determining, and then measuring, the most appropriate outcomes to accurately reflect the scope of value and benefits of health programmes for individual participants and communities. 2,5 The outcomes considered in health evaluations are limited by the perspective of those commissioning the evaluation, cost and the availability and accessibility of the data needed for the report. As such, evaluations often assess measurable clinical outcomes. 1,4 However, unlike clinical outcomes, information about how Indigenous participants experience health programmes may help to identify the specific aspects of service delivery influencing service use or engagement which is essential to achieving long-term health outcomes. 6 A recent systematic review of Indigenous health evaluations outlined experiential outcomes, not typically measured and reported such as trust and empowerment. 4 It is well established that political disenfranchisement, social and economic disadvantage and discrimination 7 have eroded trust of government institutions for Indigenous communities in high-income countries and 8,9 often excluded them when developing interventions to improve their health and quality of life. 8 Previous research has demonstrated that services which fail to implement processes that secure the trust of the Indigenous communities and empower people to achieve their health goals, fail to sustain engagement. 6 However, little is known about how widespread these experiential outcomes are in Indigenous communities to warrant consistent inclusion in evaluation reports and equally important, the facilitators and barriers to engagement with health services and ultimately the achievement of long-term health outcomes. 10 This systematic review aims to describe stakeholder (programme participant and community) perspectives and experiences with the implementation and impact of Indigenous health programmes, outlining the experiential outcomes, the reasons they are reported by respondents and the specific aspects of service delivery that influences achieving them. We acknowledge that the term 'Indigenous' is inadequate in representing the heterogeneity of the cultures and traditions of the populations described in this paper. We recognize that a more acceptable global collective term is needed.

| ME THODS
We followed the Enhancing Transparency of Reporting the Synthesis of Qualitative Research (ENTREQ) framework. 11

| Inclusion criteria
Eligible studies included primary studies, published in peer-reviewed journals where the main objective was to evaluate a complex health intervention developed specifically for an Indigenous community.
We defined complex interventions as interventions with several interacting components bringing together multiple systems and stakeholders to achieve programme delivery with multiple and multi-level outcomes versus the 'simple' interventions which focus on individual-level outcomes. 12 We included evaluations conducted after a health intervention had been developed and implemented (e.g. process evaluations, outcome evaluations and economic evaluations), as defined by the Centers for Disease Control. We excluded formative evaluations because they are part of the programme development process and we were interested in the benefits of health interventions to Indigenous communities once fully implemented. 13 Studies that reported qualitative data on the experiences and perspectives of stakeholders on the implementation and impact of Indigenous health interventions were eligible. Stakeholders included Indigenous participants in the programmes, family, staff and community members with knowledge about the health programme. Only studies of interventions among Indigenous communities from countries classified as 'high income' by the World Bank in 2017 14 were included because of the similarities in the health inequities they experience compared to the non-Indigenous populations within the same countries. Non-English studies were excluded due to lack of resources for translation.

| Data sources and searches
The search strategy is provided in Table S1. We conducted searches in MEDLINE, PreMEDLINE, Embase, PsycINFO, EconLit and CINAHL from inception to January 2020. Reference lists of relevant systematic reviews and included studies were searched for additional studies. Title and abstract screening were undertaken by SC K E Y W O R D S health policy, health services, impact evaluation, Indigenous health, outcomes to identify articles for full-text screening. An independent title and abstract screen against the inclusion criteria were undertaken by MH. Disagreements were resolved through discussion and consultation with KH. Full texts of the remaining articles were assessed for study eligibility.

| Appraisal of reporting
We critically appraised the quality of reporting for included papers using the Consolidated Criteria for Reporting Qualitative Research (COREQ). 15 This framework included criteria specific to the research team, study methods, context of the study, analysis and interpretations. The studies were independently assessed by authors SC and JD. Disagreements were resolved through discussion.

| Data analysis
We used thematic analysis to analyse data from the qualitative studies following guidelines from Thomas

| Literature search
We included 28 studies involving more than 677 participants (seven studies did not report the number of participants) from Indigenous populations in four countries: Canada (36% studies), Australia (29%), United States (25%) and New Zealand (11%) (Figure 1). Stakeholders  Table 1.

| Comprehensiveness of reporting
The comprehensiveness of reporting was highly variable with studies reporting 0 to 25 items on the COREQ checklist (Table 2).
Eighteen (64%) studies reported on the interviewer or facilitator, 18 (64%) provided information on the methodology and/or theoretical framework used, 22 (79%) provided details on the sample size, 17 (61%) reported on the method of approach used for participant selection, and 20 (71%) studies included participant quotations.

| Synthesis
We identified six themes: enabling engagement, regaining control of health, improving social health and belonging, preserving community and culture, cultivating hope for a better life, and threats to long-term programme viability. Each theme and sub-theme is described below with selected quotations provided in Table S2.
Concepts specific to a stakeholder group, type of intervention or setting are described accordingly. A schema to show the conceptual links between the themes is shown in Figure 2 Participants are those who participated in the health programme.

TA B L E 1 (Continued)
accessing health services particularly those related to social and economic disadvantage.

Relating to programme champions
Participants were interested in programmes that involved respected community leaders who they could relate to and who inspired them to achieve their health goals without making them feel patronized. 22

Confidence in programme methods
Community members perceived that participants felt more confident in programmes that demonstrated tangible progress. 26 One Māori community member observed that a community injury prevention programme had contributed to 'a gradual shift in mind-sets and attitudes towards not drinking and driving' 27 among members of the community as they chose to adopt healthier behaviours that promoted health and well-being.

| Regaining control of health
Participants wanted programmes which empowered them and helped them regain control over their health, as they defined it, and supported their approaches to achieving healthy lives.

Equipped for self-management
Participants appreciated programmes that taught and provided re-

Supporting self-determination
In some communities, programmes helped to dismantle and challenge beliefs that kept people from exercising their individual power and capacity to improve health outcomes, by strengthening awareness over the aspects that they could control. 'I really hadn't considered that we could be doing things to prevent injuries. I thought that accidents just happened,' (Māori participant of an injury prevention programme). 27 One staff member of a diabetes prevention programme for a Canadian First Nations community observed that participants wanted to feel supported in reframing their role in im-

| Improving social health and belonging
Participants felt some programmes helped them ameliorate the isolating effects of ill health and facilitate the process of restoring essential social connections that provide a sense of belonging.

Re-establishing family ties
Family relationships could become strained as a result of ill health and participants appreciated programmes that facilitated reconnection with family. This reconnection helped support them in their health journey, ensuring they didn't feel isolated and removed from their social networks. 'Well, it feels a lot better now… It wasn't like that when I was drinking. They look at me and they say oh he's drunk and let's go someplace else. Like when they come here they just give

| Preserving community and culture
Programme activities that aligned with efforts to maintain community and culture reassured participants of the programme's commitment to community values.

F I G U R E 2
Thematic schema. Interventions that enabled engagement by providing ease of access to health services, ensuring participants felt known, valued, safe, confident in the programme's methods to address their health condition, and included programme champions were valued. Once engaged, participants regained control of their health, experienced improved social health and sense of belonging and felt that their participation contributed towards preserving their culture and community. Participants experienced these impacts throughout the life of the programme adding to a greater sense of hope around achieving health goals. When the long-term programme viability was threatened, engagement was disrupted and confidence diminished in health programmes. The potential to achieve important individual and community outcomes was weakened as trust was eroded.

| Threats to long-term programme viability
Due to bad experiences with programmes, stakeholders had concerns about sustainability.

Apprehensive about losing programmes
Experience with programmes being cut prematurely left stakeholders feeling fearful that community members may miss out and that the cycle of disadvantage would continue. 'I've seen this many, many times in the Indian Health Service and now in the Alaska Native Health System is that when the grant for a program starts to shrink, the program also begins to shrink. And when the grant goes away, the program goes away and it has nothing to do with how important that program is to the health of the population or how successful the program is,' (key informant for an Alaskan Native cancer outreach programme). 34

Waning interest
Stakeholders perceived that in some communities, the initial interest in a health programme could be attributed to the novelty of the initiative, 'At the beginning, it was good because the program was new…' (community member of a Yup'ik Alaskan Native mental health programme). 32 However, often, participation decreased when the novelty wore off.

Undermining community trust
Stakeholders felt disappointed by short-term programmes and perceived that they eroded community trust in the health system. 'It's not a one-time effort, it has to be done over and over,' (stakeholder from a Native American tobacco cessation programme). 22

| D ISCUSS I ON
Indigenous participants experienced a complex interplay between factors affecting their engagement in health programmes and the outcomes of participation. They perceived that access to health programmes was facilitated when they felt known, valued, safe and confident in the methods used to address their health condition, and when interventions employed 'champions' they could relate to. Once engaged, participants were able to regain control of their health. They felt that their participation contributed towards preserving culture and community leading to improved social health and a greater sense of belonging. When these outcomes were successfully achieved throughout the programme, this added to a greater sense of hope about their lives. When the long-term viability of programmes was threatened, engagement was disrupted as confidence diminished and the potential to achieve the individual-and community-level outcomes were weakened with the loss of trust.
On an individual level, the key message from all Indigenous par- In this review, participants pointed to the mechanisms and processes within health programmes that facilitated achieving positive experiential outcomes. Our review supports the use of qualitative methods within process evaluations, particularly in identifying and understanding the value of the medium and long-term outcomes of health programmes. 47 Our assessment of the comprehensiveness of reporting from the evaluation studies in our review revealed poor reporting of the qualitative data collection and analysis processes which supports the need to promote the use of frameworks to improve the quality of reporting of qualitative evaluations of Indigenous health interventions and programmes. 5,47 The results from this review reveal the depth and diversity of stakeholder perspectives on participants' experiences with the im-

CO N FLI C T O F I NTE R E S T
None to declare.

AUTH O R CO NTR I B UTI O N S
SC conducted the search, screened and analysed the data and drafted the manuscript. JD independently reviewed and assessed included articles. AT, MH, MD, JC, SE and KH provided significant input into the conceptualization of the study, study design and analysis of the data. All authors critically revised and approved the final manuscript.

DATA AVA I L A B I L I T Y S TAT E M E N T
The data supporting the findings of this study are available within the article and its supplementary materials.