PROTOCOL: Guidance for stakeholder engagement in guideline development: A systematic review

Abstract This is the protocol for a Campbell systematic review. The objectives are as follows: to identify, describe, and summarize existing guidance and methods for multistakeholder engagement throughout the health guideline development process.


| The problem, condition or issue
Evidence-informed healthcare guidelines evaluate and summarize the available evidence regarding patient care, public health and health systems, weigh the benefits and risks as well as acceptability, feasibility, and potential equity considerations that accompany all care and policy options and make recommendations (IOM, 2011).
There is a moral imperative to include intended end users in the development of guidelines that may affect them. Stakeholder engagement, of all those potentially affected by the recommendations included in a guideline, is critical to ensuring the right issues are addressed and the right questions are asked (Gillard, 2012;Oliver, 2014). It may improve the relevancy, transparency and usefulness of guidelines (Esmail, 2015). Stakeholder engagement helps to ensure guideline acceptability and feasibility, support for its uptake and the practices recommended, and possible effects on adherence to any treatments and practices recommended (Carroll, 2017;Schunemann, 2014). Successful guideline development and implementation requires engagement of multiple stakeholders as well as shared solutions to improve health outcomes (Dunston, 2009;Kumarasame, 2016;Suman, 2016). Stakeholder groups potentially affected by guideline recommendations include patients, caregivers and the public, providers, payers, purchasers, product makers, policy makers, program managers, peer review editors, and principal investigators (Concannon, 2012;Tugwell, 2006). Clinicians and patients/consumers are the stakeholder groups predominantly engaged in guideline development and there has been little guidance developed for including other groups of stakeholders Lavis, 2008;van de Bovenkamp, 2015).
Effective stakeholder engagement should include the equitable inclusion of different groups and those who are known to be underrepresented. We use the term 'underrepresented' to refer to those individuals or groups who are (1) typically excluded from guideline development and implementation and (2) those who may experience inequities. These health inequities may exist for reasons such as a lack of inclusion in research, barriers to access of services, or because of other socially stratifying factors (Shi, 2014;Wallerstein, 2010). These factors can be described using the acronym PROGRESS-Plus (O'Neill, 2014); Place of residence, Race/ethnicity/ culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital, or other characteristic, such as age, in the guideline development and implementation process. Engagement of those affected by the guidelines and their recommendations ensures equity, diversity, and inclusion as well as social justice issues as it democratizes the guideline development process. It allows for stakeholder to participate in decision making for recommendations, programs, and policies that affect them.
There are many existing frameworks for how to develop guidelines, however, stakeholder engagement is often included as one step of the process and guidance for stakeholder engagement throughout the individual steps of the process may be lacking. Schunemann (2014) identified 18 topics in the guideline development process. Stakeholder engagement is one topic on the checklist (item 6). Selva et al. (2017) reviewed 56 guidance documents for guideline development and found that while 72% included the engagement of patients and their views in the guideline development process, few provided details on how it may be done. In light of the importance of stakeholder engagement in guideline development, this review aims to identify and describe existing guidance for stakeholder engagement that have been or that can be employed at different stages in the guideline development process. This includes identifying who to engage, when to engage them, and how to involve them throughout the process.

| Definitions
For our purposes, • Guidelines are 'systematically developed evidence-based statements which assist providers, recipients and other stakeholders informed decisions about appropriate health interventions' (WHO, 2003).
• Stakeholders include patients, caregivers and the public, providers, payers of health services, payers of research, purchasers, product makers, policy makers, program managers, peer review editors, principal investigators or anyone who may be affected by a guideline.
• engagement entails an approach to ensure the contribution of stakeholders towards the development of the guideline, completion of any of the stages of the guideline, or dissemination of the guideline and its recommendations (Frank, 2020;. It is important to acknowledge that language in this field varies internationally and is continually evolving. Terms such as involvement, collaboration, or partnership have also been used to refer to engagement (Hoddinott, 2018).
Within this review we use the term 'engagement'.
• existing guidance describe systematic approaches for stakeholder engagement ) and we will include descriptions of process(es), checklists, concepts, models, outlines, systems, plans and or overviews on engaging stakeholders in guideline development processes.

| Description of the phenomena of interest
The role and value of stakeholder engagement in guideline development is increasing and therefore there is a need to identify and describe the existing guidance as this information will be useful for creating checklists for the meaningful engagement of multiple stakeholder groups in the guideline development process.
OIiver et al. have described that for committees, such as those involved in guideline development, multispeciality groups are recommended over singly speciality groups to allow for a greater range of opinion. While larger groups may be more difficult to manage and require a skilled and experienced facilitator, they also offer more opportunity for diversity among members and therefore may be more reliable, may enhance credibility and lead to widespread acceptance and implementation of decisions and recommendations .
Stakeholder engagement may occur at various steps in the process. Stakeholder intensity can vary with some stakeholder groups being involved more intensely at certain points in the process (Crowe, 2017;Oliver, 2008;Pollock, 2019). The input of different stakeholder groups at different steps of the guideline process may have different affects on the development process. For example, including peer review editors early in the process ensures that the final guidance document is written according to standards required by the journals that may publish it, including patients during question formulation and the selection of outcomes ensures that their concerns are addresses, including program managers in steps related to evaluation and use ensures that the guideline and recommendations can be evaluated. In addition, they can be involved in advisory or feedback roles, where they receive information and are invited to provide their opinions and/or experience but for which there is no commitment to act on the feedback provided, or they may be involved in decision-making in which their views influence the overall content of the guidelines. Engagement requires certain activities to be effective ( Figure 1). Stakeholder engagement is a complex process and there are many factors that may influence stakeholders' ability to engage in the guideline development process as well as barriers to guideline developer's abilities to facilitate engagement. This scoping review focuses on existing guidance for how to effectively engage stakeholders in the process as described by the activities listed in our logic model ( Figure 1) and mapped to the 18 topics of the GIN-McMaster checklist.

| Why it is important to do the review
Guidance for guideline development, including stakeholder engagement, exists. Many guideline development organizations provide guidance for guideline development but do not provide specific information about how and when to engage multiple stakeholders.
The Guidelines International Network (GIN) has proposed key elements, including stakeholder engagement, that need to be considered in developing guidelines (Qaseem, 2012). In identifying components for guidelines on guideline development, Schunemann and colleagues   | 3 of 10 health equity, equality, and nondiscrimination, societal implications, financial and economic considerations, feasibility and health system considerations) and suggested methods to facilitate uptake of recommendations (Rehfuess, 2018 (Schunemann, 2014). This overview of guidance manuals is currently being updated. However, it does not specifically focus on stakeholder engagement throughout each step of the guideline development process.
In engaging stakeholders, many guideline developers focus on a single stakeholder, particularly patients, caregivers, or service users (e.g., persons without lived experience of the medical condition related to the guideline). For example, Armstrong and colleagues have developed a 10-step model for engaging patients in guideline development. . We aim to additionally consider guidance for the interactions between multiple stakeholder groups on guideline development and implementation.
The GRADE (Grading of Recommendations Assessment, Development and Evaluation) system is internationally recognized as a standard for guideline development (Guyatt, 2008). The Multi-Stakeholder Engagement (MuSE) working group is a global consortium established in 2015 to look into the development of methods for involving multiple stakeholders in health outcomes research (Concannon, 2019;Petkovic, 2020). One goal is to develop guidance for multistakeholder engagement in guideline development. To develop this guidance, the group will conduct this systematic review to identify existing guidance for stakeholder engagement in guideline development in parallel with three other reviews focused on (

| OBJECTIVES
The objective of this review is to identify, describe, and summarize existing guidance and methods for multistakeholder engagement throughout the health guideline development process.

| Types of study designs
Our methods will follow the Cochrane Handbook for Systematic Reviews of Interventions and the Handbook for Qualitative Research, as appropriate (Higgins, 2019;Sandelowski, 2007).
To be included, papers must describe the process or methods for stakeholder engagement in guideline development using the definition of guideline described above. We will include quantitative, qualitative, and mixed-method studies. Eligible study designs include: randomized controlled trials (RCTs), quasi-randomized controlled trials, and nonrandomized studies (e.g., before and after studies, cohort studies, cross-sectional studies) as well as theoretical papers, process evaluation studies, and qualitative studies. Mixed methods studies that apply a combination of the eligible quantitative and qualitative study designs and report on qualitative and quantitative outcomes separately will be eligible.
We will exclude editorials, commentaries, protocols, and conference abstracts. We will also exclude guidance provided in handbooks

| Population of interest
We have identified 13 types of stakeholders whose input can enhance the relevance and uptake of research (Concannon, 2012(Concannon, , 2019Tugwell, 2006 (Hoddinott, 2018) have also been used in reference to engagement.
Herein, we will use the term 'stakeholder engagement'.
As an example, a potentially included study describes the development of a 10-step framework for patient engagement in clinical practice guideline development .

| Phenomena of interest
We will focus on studies that describe stakeholder engagement in the guideline development process. We will include studies describing the development and/or implementation of a process for stakeholder engagement in guideline development. Guidelines on healthcarerelated issues, and clinical, public, or social health will be considered.
We will include papers discussing any topic of the guideline • Level of engagement (Crowe, 2017;Oliver, 2008;Pollock, 2019) in each step of the guideline development process (advisory/feedback, decision-making) • How stakeholders contributed at each step of the guideline development process • Methods for resolving disagreements • Evaluation of engagement processes

| Types of settings
We will place no restrictions on setting.

| Search strategy
We will develop one comprehensive search strategy for all four systematic reviews in consultation with a medical librarian which will be reviewed by a second medical librarian. We will search the following databases: MEDLINE (OVID), CINAHL (EBSCO), EMBASE (OVID), PsycInfo (OVID) and SCOPUS. Limits will not be placed on date, study design or language.
We will conduct an extensive grey literature search. We will search the websites of agencies who actively engage stakeholder groups in their work such as the Agency for Healthcare Research and

| Description of methods used in primary research
It is expected that included papers will describe frameworks or guidance for who, when and how to engage stakeholders in guideline development and implementation.

| Details of study coding categories
Two reviewers will independently screen titles and abstracts to identify relevant studies meeting the pre-specified inclusion criteria. The full text of potentially included studies will be screened independently by two authors. We will use Covidence software (https://www.covidence.org/) for screening of studies. All completed studies will be included if they meet the inclusion criteria listed above.
The data extraction form will be pre-tested and extracted independently in duplicate by two reviewers using Eppi-Reviewer to facilitate qualitative synthesis (described below) (Thomas, 2010).
Disagreements will be resolved by discussion and with a third member of the research team when necessary.
We will extract details related to guidance for stakeholder engagement for each of our identified stakeholder groups for each of the 18 topics (146 steps) of the GIN-McMaster checklist (Table 1).
We will also extract data on: • General study characteristics such as study design and year of publication • Stakeholder groups and definition of stakeholder

| Quality assessment
We will assess the quality of our included papers using the criteria: practicality, relevance, and legitimacy as described in the tool developed by Movsisyan et al. (2019). This tool assesses, for example, whether the paper in describes key concepts clearly, whether the guidance can be easily applied without additional information, whether there is information related to its adaptation to other settings, and whether the underlying theory and principles have been described (see Table 2). T A B L E 1 (Continued) We will also assess whether the development of the guidance has included input from relevant stakeholders.
Quality will be assessed independently, in duplicate, by two authors and any discrepancies will be resolved by consensus and consultation with a third author, when necessary.

| Statistical procedures and conventions
We will not have data for statistical analysis.

| Treatment of qualitative research
We will narratively summarize the existing guidance identified in our included studies by mapping the descriptions of stakeholder engagement in the included papers to the 18 topics (146 steps) of the guideline development process outlined by the GIN-McMaster checklist (Schunemann, 2014). We will present this information as a matrix indicating the guidance that exists for each stakeholder group and for each step of guideline development.
We will use thematic synthesis to combine guidance for stakeholder involvement in each step.
We will follow the guidance for qualitative synthesis in systematic reviews as outlined by Thomas (2008) which includes coding the text and developing descriptive themes (Thomas, 2008).
This involves translating concepts from one study to another and looking for similarities and differences between codes to allow for grouping concepts into themes for analysis (Thomas, 2008).

SOURCES OF SUPPORT
Internal sources Are the key items/guidance clearly specified?
Ease of use and operationalisability −/+/++ Guiding questions: Can the guidance be applied easily without the need to search for additional information?
Can the guidance be adequately operationalised?
Comprehensiveness −/+/++ Guiding questions: Does the guidance thoroughly describe information related to adaptation in a new setting?
Does the guidance thoroughly describe information related to evaluation in a new setting?

Relevance
Relevance for use in different settings and with different stakeholders −/+/++ Guiding question: Can the guidance be applied to different types of guidelines?
Can the guidance be used in different settings?
Can the guidance be adapted to additional stakeholders?

Legitimacy
Legitimacy-Scientific basis and development process −/+/++ Guiding questions: Does the guidance describe its underlying theory and principles?
Does the guidance describe a rigorous development process (such as a comprehensive literature review and/or a consensus-based methodology) Abbreviations: −, not addressed; +, partially addressed; ++, fully addressed.
• No sources of support provided Foundation that address a similar topic.