Exploring the impact of engagement in mental health and substance use research: A scoping review and thematic analysis

Abstract Background There is growing evidence demonstrating the impact of engaging people with lived experience (PWLE) in health research. However, it remains unclear what evidence is available regarding the impact of engagement specific to mental health and substance use research. Methods A scoping review of three databases and thematic analysis were conducted. Sixty‐one articles that described the impact of engagement in mental health and substance use research on either individual experiences or the research process were included. Results Key topics include (a) the impact of engagement on individual experiences; (b) the impact of engagement on the research process; and (c) facilitators and barriers to impactful engagement. Studies largely focused on the perceived positive impact of engagement on PWLE (e.g., personal and professional growth, empowering and rewarding experience, feeling heard and valued), researchers (e.g., rewarding experience, deeper understanding of research topic, changes to practice), and study participants (e.g., added value, fostered a safe space). Engagement activities were perceived to improve facets of the research process, such as improvements to research quality (e.g., rigour, trustworthiness, relevance to the community), research components (e.g., recruitment), and the research environment (e.g., shifted power dynamics). Facilitators and barriers were mapped onto the lived experience, researcher, team, and institutional levels. Commonly used terminologies for engagement and PWLE were discussed. Conclusion Engaging PWLE—from consultation to co‐creation throughout the research cycle—is perceived as having a positive impact on both the research process and individual experiences. Future research is needed to bring consistency to engagement, leverage the facilitators to engagement, and address the barriers, and in turn generate research findings that have value not only to the scientific community, but also to the people impacted by the science. Patient or Public Contribution PWLE were engaged throughout the scoping review process, including the screening phase, analysis phase, and write‐up phase.


| INTRODUCTION
The past two decades have demonstrated a shift towards engaging people with lived experience (PWLE) in health research as collaborators, rather than as study participants. 1 Often referred to as patient engagement, consumer engagement, patient and public involvement, or co-production, engagement in research involves authentic and ongoing collaboration with PWLE across the research cycle, from conceptualization to dissemination. 2 PWLE can be engaged on a continuum, ranging from consultation and advisory roles to equal partnerships, leadership, and decision-making roles. 3 Engagement in research has been framed as a way to improve research quality and relevance of study findings to the community, in addition to being an ethical imperative. 4 Funding bodies are increasingly interested in supporting researchers who engage PWLE throughout the research process to improve the impact, quality, and relevance of the research they fund. [5][6][7] For instance, institutions such as the Centre for Engagement and Dissemination in the United Kingdom, 8

Patient-Centered
Outcomes Research Institute in the United States, 6

and the Canadian
Institutes of Health Research Strategy for Patient-Oriented Research in Canada 2 have set guidelines and policies around engagement in health research. As institutions continue to set national standards around engagement, efforts should be made to mitigate potential harms from tokenistic and inequitable practices. 9 This is especially important as engagement in research is often critiqued for having limited representation of socially marginalized groups (e.g., racialization, low income) among collaborators. 10,11 The shift from passive recipients to active experts, researchers, and leaders has fueled an expanding body of democratic research in the mental health and substance use field. The moral obligation of engagement is especially relevant to this field given the historical oppression and coercive practices in psychiatry that have left patients silenced, without power, and as passive recipients of care. 12 Grassroots movements in the late twentieth century played a key role in dismantling power dynamics and challenging current practices in psychiatry, along with advocating for the involvement of PWLE as key knowledgemakers within systems restructuring and research. 12, 13 Yet, there is work to do to continue unpacking past practices, especially as ongoing progress is often overshadowed by research priorities rather than challenging power structures within institutions. 14 There is growing evidence demonstrating the impact of engagement in health research. Previous engagement research suggests that the way impact is measured and reported is inconsistent and limited to subjective accounts of impacts. [15][16][17] Evidence from reviews that assess impact varies regarding the impact on who, such as the impact on individuals (e.g., PWLE and researchers) 18 or what, such as the impact on research design and delivery (e.g., recruitment rates in clinical trials). 19,20 Similar to how evidence-based medicine is viewed as the gold standard in health research, current practices in health research largely focused on measuring the impact of engagement on the research process. 9 However, especially in the mental health field, framing engagement as solely valuable to the research itself risks undermining the ethical imperative behind engagement activities. 21 Recent reviews have described the impact of engagement in health research; 18,20,22 however, it remains unclear what evidence is available regarding the impact of engagement specific to mental health and substance use research. Indeed, the assumptions underlying impact in health research may not apply to a mental health and substance use context given the abundant presence of power imbalances, the perceived vulnerability of PWLE, and the stigma around the capacity of PWLE to consult in research projects. 12,13 Therefore, this scoping review aims to map the literature on how the impact of engagement is conceived in mental health and substance use research. A scoping review is ideal for this study as the topic is an emerging field, with a wide range of designs across studies, allowing for a flexible approachan important feature as the state of the evidence and methods is unclear. 23 The specific objectives of this review are to map the

| METHODS
The present scoping review was guided by Arksey and O'Malley's framework 24 and the enhanced framework by Levac and colleagues. 25 The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews checklist was followed to ensure methodological and reporting quality. 26  Patient participation/OR (patient* or client* or public or 'service user*' or youth or consumer* or citizen*) adj2 (participat* or engag* or invol*) OR 'liv* expertise' or 'lived experience' or 'peer* researcher*' or 'co-researcher*' or 'expert* by experience*' or 'patient* partner*' or 'patient* advisor*' or 'co-produc*' or 'co-design' Mental Health/OR (mental* or psychiatr* or psycholog*) adj2 (health* or ill* or hygiene or disorder* or distress*) OR (drug* or substance* or alcohol*) adj2 (abus* or addict* or depend* or misus* or use* or dependen* or disorder*) (improv* or strength* or inform* or increase* or impact* or facilitat* or support*) adj3 (research* or method* or design or outcome* or recruit* or study or team*) Databases Medline (Ovid), CINAHL (EBSCO), and PsycINFO (ProQuest) Inclusion (a) academic journal articles, including full-length original research (e.g., qualitative, quantitative, mixed or multi-methods, case studies), brief reports, or commentaries/viewpoints that provide an overview of the engagement process in a project  A full description of search terms and strategies is shown in Table 1. As 'patient engagement' is broadly defined, without consistent terminology, multiple search strategies were piloted to identify appropriate keywords. After an initial search was conducted to determine which terms reflect the phenomena in the research question, the following combination of search terms were used to broadly define PWLE, such as patient*, client*, public, service user*, youth, consumer*, and citizen*. To capture engagement, search terms such as engag*, participat*, and invol* were used adjacent to the term for PWLE, in addition to other terms such as co-researcher*, codesign*, and co-produc*. Search terms such as mental, psychiatr*, psycholog*, substance* were used adjacent to health*, use*, disorder* to situate the search in the field of mental health and substance use.
Lastly, search terms such as impact*, support*, and improv* were included to capture studies that discussed the impact. The search terms were adapted for each database concerning the proximity operators, truncations, and wildcard symbols. Facilitators and barriers were added as data items during the pilot data charting process due to the dominance of these factors in the selected articles and the richness of information they provided.

| Selecting sources of evidence
In addition to the data charting tool, the 61 articles were uploaded to NVivo 12 and analysed by a single coder (NYS) using the codebook approach to thematic analysis. 30 Here, themes are conceptualized as topic summaries. 30 An initial coding framework was developed by NYS based on the piloted data charting process.
Throughout the process, additional codes and themes were developed inductively and refined through weekly meetings with two of the authors (NYS, LDH). The codes and themes were further refined through feedback from a larger research team within the same unit, of which many team members identified as having lived experience, in addition to the unit's Lived Experience Advisory Group. Lastly, using the text frequency option on NVivo 12, we explored the most commonly used terminologies for 'engagement' and 'people with lived experience'. This was added at a later phase as a separate analysis from the thematic analysis and was based on the feedback from the Lived Experience Advisory Group.

| Synthesis of results
The extracted data from Excel and NVivo were collated and summarized in a narrated format according to the key outcomes

| Impact of engagement on individual experiences (N = 48)
Perceived impacts related to individual experiences were described in 48 articles ( Figure 2). Here, 35  3.2 | Impact on the research process (N = 50) Fifty studies described the impact of engagement on the research process ( Figure 2

| Facilitators and barriers to impactful engagement
The majority (N = 51) of the studies identified facilitators and barriers to impactful engagement in mental health and substance use research (

| Commonly used terminologies
The most commonly used terminologies for engagement and PWLE are shown in Table 4. Out of 61 studies, commonly used terminology for engagement across the included studies was 'participatory

| Epistemic versus ethical justification for impact
Our findings indicate similarities between mental health, substance use, and other health research fields regarding the perceived impacts on the research process. For instance, reviews in health research have reported the positive impact of engagement on research design, study recruitment, 93 and data collection. 6,93,94 These findings demonstrate an emphasis on epistemic benefits, such as the 3R's of research-rigour (e.g., rigour related to study design), relevance (e.g., relevance to population needs), and reach (e.g., dissemination and knowledge translation). 95 Note, engagement in mental health research is critiqued for overemphasizing epistemic benefits rather than ethical ones. 21 If research teams focus solely on impacts related to epistemic benefits while neglecting ethical imperatives, the justification for engagement risks being treated as a means to an end of achieving better research, rather than being an intrinsically good and democratic process. 96 Much like the studies included in our review, the broader health literature focuses on process indicators and the perceived impact of engagement rather than empirical outcomes. 6 that quantitative approaches are less useful for evaluating the impact of engagement given its complex and context-dependent nature. 94

| Evidence-based medicine and power
The understanding of impact within academic institutions remains heavily influenced by evidence-based medicine, which at times can be counterintuitive to engagement as it undervalues the needs of PWLE. 99 Traditionally, evidence-based medicine has been considered to devalue lived-experience knowledge by perpetuating evidence hierarchies and failing to address power imbalances (especially apparent in the mental health field). 100   can promote transparency and consistency when reporting on the impact of engagement in a study. 16,102 However, the GRIPP2 checklist is not specific to mental health and substance use research, which may explain why it was followed in only 8 of the 61 articles included in this review. The use of reporting guidelines for engagement in mental health and substance use research should be further explored, including the development of a guideline specifically tailored to the unique characteristics and challenges of this field.

| Individual experiences
Without consistent reporting, a fulsome understanding of the impact of engagement in mental health and substance use researchwhether on individuals or the research process-will remain unclear.

| Strengths and limitations
The strengths of this study include the use of multiple databases, a comprehensive search strategy, and thematic analysis, in addition to engaging with PWLE throughout the scoping review process.
However, several limitations warrant consideration. First, the review was limited to academic journal articles; it is possible that the grey literature could include facets missing in the academic press related to impact. Future reviews may benefit from expanding the search strategy to include the grey literature. Second, the articles came from high-income countries and were only in English; therefore, they may not be generalizable to other settings. Third, the review may have missed relevant articles given the wide range of terminology related to engagement in mental health and substance use research.
Similarly, articles with PWLE as researchers were not explored (e.g., survivor-led research), which is an important topic for future research. Fourth, the inconsistent and largely subjective impacts reported across the literature may have influenced the study findings.
Future research may benefit from standardized reporting of engagement impacts for reproducibility; however, this should not overshadow the ethical imperatives for engagement. Lastly, no assessment of the risk of bias or critical appraisal was conducted for this review, although this is not required for scoping reviews. 26

| CONCLUSION
This scoping review presents a comprehensive overview of the current literature on engagement impacts in mental health and substance use research. Additionally, facilitators and barriers to impactful engagement were identified at the lived experience, researcher, team, and institutional levels, which can be leveraged to improve engagement practices.
Engaging PWLE throughout the research cycle-from consultation to co-creation-was perceived as having a positive impact on both the research process and individual experiences. Future research is needed to bring consistency to engagement, leverage the facilitators to engagement, and address the barriers such as power differences. This would in turn generate research findings that have value to the scientific community, but also to the people impacted by the science. Sheikhan. All authors read and approved the final manuscript.