Participation and consultation engagement strategies have complementary roles: A case study of patient and public involvement in clinical practice guideline development

Abstract Background Patient and public involvement (PPI) is recommended when developing high‐quality clinical practice guidelines, but the effects of different PPI strategies are largely unstudied. Objective To assess the impact of participation and consultation strategies on guideline question development. Design Instrumental case study design. Setting and participants This study used a clinical practice guideline in development by the American Academy of Neurology. A patient, two caregivers and a dementia advocate participated in the guideline development group alongside clinicians. The guideline protocol was posted for public consultation for 30 days. Interventions studied Participation (patient representatives on the guideline development group) and consultation (public comment, survey) PPI strategies. Main outcome measures Public comment responses and guideline development group meeting transcripts were analysed descriptively. Transcript quotes were compared to the conceptual model of PPI in guideline development. The effects of participation and consultation strategies within the guideline case were compared. Results Participation strategies shaped discussions, set a patient‐centred scope, highlighted personal aspects of disease, affected how professionals viewed PPI, identified issues overlooked by medical professionals, and contributed to selecting patient‐relevant guideline populations and outcomes. Professionals responded to public comment more than patient representatives. Patient survey participants confirmed the priorities voiced by patient representatives on the guideline development group. Final guideline questions included populations and outcomes promoted by patient representatives despite negative feedback from professional public commenters. Discussion and conclusions Participation and consultation PPI strategies have different advantages. Congruence between strategies increases the strength of the patient voice. Guideline developers should prioritize using both strategies for successful PPI.


| INTRODUC TI ON
Excellence. 3 Both the Appraisal of Guidelines for Research and Evaluation II instrument 4

and the Guideline Trustworthiness,
Relevance and Utility Scoring Tool 5 assess diversity in stakeholder involvement, including the presence of patient representatives. PPI in guidelines is advocated because it leads to the development of more patient-centred and trustworthy guidelines, recognizes patients as experts, respects citizen rights in developing health policy, and empowers and informs consumers making health-care decisions. 6 Consultation and participation are the two primary mechanisms for PPI in guidelines. 7 These strategies are also part of the patient engagement continuum in the framework for patient engagement in health-care policy. 8 PPI mechanisms are characterized by degree of participant involvement, direction of information flow and representativeness of the population engaged ( Figure 1). Consultation involves a unidirectional flow of information and opinions from patients and the public to a guideline development group, often through focus groups, surveys or public comment. With consultation strategies, there is typically no back-and-forth interaction between patient and public stakeholders and the guideline development group. In contrast, participation is achieved by including one or more patient or public stakeholders on a guideline development group alongside professional members (Figure 1). There is a bidirectional exchange of information, allowing all stakeholders to actively participate in deliberations and fostering mutual influence between stakeholder types.
There is also development of a collective perspective. [6][7][8] Consultation and participation strategies have distinct strengths and limitations. Consultation strategies collect a variety of perspectives from a large group of people, but fail to recognize patient and public representatives as development partners or give them an active voice in the process. The views or suggestions offered through consultation may not undergo deliberation or inform decision making. Participation strategies recognize the unique expertise of patients and the public and facilitate mutual learning and compromise. However, they rely on involvement of a small number of representatives, potentially missing the perspectives of uninvolved parties. 7 A taxonomy for stakeholder engagement in patient-centred outcomes research defined engagement as involving a bidirectional relationship, 9 thus excluding consultation strategies. However, guideline developers report that recruitment difficulties, 10 representativeness of selected participants, 10 training and support needs, 3,11,12 uncertainty of how to incorporate patient experiences, 11,13 patient representatives' feelings of isolation, 10 and difficulty with medical terminology and systematic review participation 1,3,10-12 are all barriers to successful participation. Employing multiple engagement strategies in guideline development may allow developers to capitalize on the strengths of each approach and avoid the limitations inherent in a single strategy, 7,14 but insufficient resources are also a known barrier to PPI in guideline development 10,11 and could limit developer willingness to select certain or multiple strategies.
Given that minimal research has evaluated the relative contributions of different PPI strategies to guideline development, we aimed to assess the effect of participation and consultation on guideline question development. The goal was to identify the impact that participation and consultation strategies had on the first guideline step-question development-for a single guideline using multiple PPI approaches, in order to inform guideline developers considering incorporating PPI in their processes. Here, we report the final results of the overarching study, including previously undescribed results of public comment (consultation), determination of final guideline questions by a guideline development group including patient representatives (participation), and an assessment of the impact of participation and consultation on guideline question development. However, the scans cannot diagnose the clinical condition of patient voice. Guideline developers should prioritize using both strategies for successful PPI.

K E Y W O R D S
amyloid PET imaging, clinical practice guidelines, dementia, guideline adherence, guidelines as topic, patient and public involvement, patient participation, patient-centred care dementia. Research shows that people desire amyloid testing results even with uncertain implications, 15,16 but consensus-based appropriate use criteria recommend limiting scans to individuals with unexplained mild cognitive impairment (MCI), possible AD dementia with unusual features or an unusually early dementia onset. 17 The guideline was chosen from the AAN's waiting list of nominated and prioritized guideline projects. 18 The US-based AAN takes a patient/stakeholder approach to PPI rather than seeking public/consumer representatives.
The overarching study design was a multiple-methods approach ( Figure 2). In Step 1, neurologists were randomly assigned  Participation benefits and harms were largely similar between groups, but only the group with PPI proposed outcomes relating to progression rates and developing cognitive impairment at specific future time points (Table S1). PPI influenced meeting conduct, guideline scope, inclusion of patient-relevant topics, outcome selection and planned approaches to recommendation development, implementation and dissemination. 20 In Step 2 ( Figure 2 Steps 3 (public comment, a consultation strategy) and 4 (finalization of guideline questions, a participation strategy) are described further below as part of the current case study approach and analysis.

| Step 3: public comment
Posting of guideline protocols for a 30-day public comment period is part of standard AAN guideline methodology 18

| Analysis
Data collected included public comment responses and the tran- This model divides the outcomes of PPI in guideline development into organizational (developer) and guideline-specific outcomes.
Organizational/developer outcomes include: • Culture of patient-centredness (shaping how discussions are conducted, setting patient-centred scope, highlighting personal aspect of disease and impacting how professional team members view PPI); and • Meaningful and effective partnerships.
Guideline-specific outcomes include: • Guidelines relevant to patients, stakeholders (identifying issues that may be overlooked by medical professionals, helping select patient-relevant topics and outcomes, and influencing guideline structure/development, including language and recommendations); and • Facilitating dissemination and implementation (education and support tools for patients and caregivers, contributing to patient guideline versions and encouraging shared decision making and active dissemination). 20 Results were reviewed and discussed with co-investigators. The impacts (as per the conceptual model) of PPI through the public comment (consultation) and final question development (participation) were compared to the impacts reported from earlier analyses regarding initial question development (participation) and the survey (consultation).

| Ethics
All question development group and survey participants consented to study activities. Public comment responses were summarized as part of the standard guideline development process.

| Public comment results (Step 3)
Fifty-four individuals responded during public comment: 35 neurologists, one neuroradiologist, one research scientist and 17 individuals who responded anonymously and declined to identify their background. At least two anonymous respondents were patient or public stakeholders, as two public comment respondents participating in the survey were identified as patient stakeholders. A group of clinician-researchers also sent a letter to the AAN during the public comment period. Because patient stakeholders did not identify themselves in the online public comment form, their responses could not be compared to those of professional respondents.
Most public comments relating to the PICOT questions focused on the proposed populations and outcomes. Multiple respondents voiced concern about the inclusion of 'pre-clinical' (asymptomatic) populations without cognitive symptoms, stating that amyloid PET is not approved for this population, it is outside the scope of the appropriate use criteria, 17 and such testing is 'ahead of its time'.
Respondents advised highlighting MCI subtypes and specifying subgroups such as individuals with known ApoE genetic status (associated with AD risk), various dementia subtypes and dementia presentations complicated by depression or comorbid conditions.
For outcomes, public comment respondents expressed concern regarding the PICOT questions looking at the prognostic (rather than diagnostic) value of amyloid PET based on neuropathologic reasons (amyloid levels plateau in early stages of AD), the perceived lack of data to answer these questions and differences in individual progression rates that may not be captured in group-level data. Other comments covered topics such as using alternate diagnostic modalities, clarifying methods of determining a 'positive' scan, emphasizing the value of a negative scan for diagnostic purposes and better framing the comparator. One respondent questioned the use of amyloid PET at all given the lack of disease-modifying therapies for AD.
Another questioned the value of an evidence-based guideline when consensus-based guidelines already exist.

| Results of second guideline question development meeting (Step 4)
Of 16 question development group members and two methodologists involved in the original retreat (n = 18), 12 question development group members and both methodologists attended the second question development retreat in person (n = 14). One content expert and one guideline subcommittee member attended by teleconference ( of the sessions with Mayo doctors wanted this test, and the concern was the cost. And I've been on [committee name], and that is, every time I talk about this, the PET scan, they said, 'But the cost'. So that's a factor in terms of who's going to pay for this. It was also a factor for the Center for Memory and Aging as to, okay, if Medicare is going to pay for this, but who's going to pay for the time to analyze it with our doctors and so forth? So cost is definitely a factor. (Caregiver 2) I know for a fact that my husband's lifestyle has made a difference in his living with the disease, and we're now in the 23rd year of his living with the disease, and he is still functioning. He's still functioning. He's at home and he needs his caregiver at home, but his daughter has him today. But it does, lifestyle does make a huge, huge difference. (Caregiver 2) Helping select patient-relevant topics and outcomes From my point of view as a person with the disease, to me, I would far rather start out on a treatment for something and find out that I didn't need it than find out later that I could have had some treatment that would have slowed a disease like this that would destroy my ability to function. So the need to know far outweighs any concerns about false positives to me (Patient) It may allay some fears from the public if we're at least trying to-showing that we're considering this issue that even if [amyloid PET] is predictive that we would [also] like to know how it actually impacts people's lives. (Expert 2) [Knowing expected prognosis] is critical for sustaining, sustenance of emotional health (Caregiver 2) Right, so, I think there's, you know, there's the additional thing, you know, that and I think we clinicians maybe undervalue this, but the value of the label for the patient may not be actionable on our part but that might have-this gets a quality of life and outcomes as well (Expert 2) the question development group opted to keep guideline questions regarding the use of amyloid PET in asymptomatic populations, even though these populations are outside the appropriate use guidelines, 17 of low value to physician survey participants 21 and recommended against by professional public comment respondents.
PPI via participation (Figure 2, Steps 1 and 4) and consultation using a survey (Figure 2, Step 2) resulted in both similar and distinct benefits (Table 3). Given the pre-specified questions and unidirectional nature of the survey (provision of opinions from survey respondents to the question development group), survey responses could not highlight personal experiences, actively shape in-person discussions or build meaningful and effective partnerships. Survey responses did, however, have benefits overlapping with those offered by participation (Table 3). Additionally, the views of the large group of patient stakeholders responding to the survey confirmed the views voiced by the four question development group participants.

| Limitations of study PPI mechanisms
There were several limitations and barriers to PPI in this case study.
In the second guideline retreat, the patient representatives raised several topics not clearly linked to the guideline. These included the patient's perceived distinction between memory and cognition and the influence of lifestyle changes on cognition and dementia. While these topics could be seen as patient stakeholders identifying issues that may be overlooked by medical professionals (Table 2), they could also be interpreted as an unnecessary use of time. Professional panellists also used a small amount of retreat time to answer the patient representatives' technical questions.
Developer time was dedicated to creating a patient-and publicfriendly guideline protocol to post for public comment, but the majority of public comment respondents were identified as professionals. The survey was a more successful consultation strategy but required extra resources for development and dissemination.

| D ISCUSS I ON
Conduct of the second guideline development retreat group confirmed the previously reported benefits of participation PPI strategies including shaping how discussions are conducted, setting a patientcentred scope, highlighting the personal aspects of disease, affecting how professional team members view PPI, identifying issues that might be overlooked by medical professionals and selecting patientrelevant guideline topics and outcomes. 20 Public comment as a consultation strategy affected discussions and final question decisions, but reflected input from professionals more than the public or patient representatives. Use of a survey as a consultation strategy, however, Participation and survey consultation approaches to PPI resulted in similar but distinct benefits ( This study employed multiple strategies to overcome known barriers to successful PPI, which may affect study generalizability.
Investigators used the patient and service user engagement in research framework 25 and pilot focus groups 26

| CON CLUS IONS
This study showed that while public comment was not successful

E TH I C A L A PPROVA L
The University of Florida Institutional Review Board provided approval for the study (IRB201501210).