Examining how study teams manage different viewpoints and priorities in patient‐centered outcomes research: Results of an embedded multiple case study

Abstract Introduction Limited evidence exists about which patient and stakeholder engagement practices support or hinder study teams as they negotiate different viewpoints in decisions about the design and conduct of patient‐centered outcomes research. Methods We applied a multiple‐embedded descriptive case study design for six studies funded by the Patient‐Centered Outcomes Research Institute (PCORI). We interviewed 32 researchers and stakeholder partners, including patients, caregivers and clinicians, and reviewed documents related to each study (e.g., publications, and progress reports submitted to PCORI). Findings Overall, researchers reported that incorporating different viewpoints was a strength or opportunity to learn rather than something to be avoided or dreaded. Across cases, different viewpoints and priorities, often related to ethical or pragmatic considerations, emerged between researchers and stakeholders, between stakeholder groups (e.g., patients and clinicians) or within groups (e.g., amongst researchers). Examples of navigating different viewpoints arose across study phases. The length of time to resolve issues depended on how strongly people disagreed and the perceived importance or impact of decisions on the study. All cases used collaborative decision‐making approaches, often described as consensus, throughout the study. Interviewees described consensus as using negotiation, compromise or working towards an agreeable decision. To encourage consensus, cases actively facilitated group discussions with an openness to diverse opinions, remained flexible and open to trying new things, referenced a ground rule or common goal and delegated decisions to partners or smaller workgroups. When viewpoints were not easily resolved, cases used different approaches to reach final decisions while maintaining relationships with partners, such as elevating decisions to leadership or agreeing to test out an approach. No one engagement structure (e.g., advisory group, coinvestigator) stood out as better able to manage different viewpoints. Teams adjusted engagement structures and behaviours to facilitate an overall culture of inclusion and respect. Partners acknowledged the intentional efforts of researchers to incorporate their perspectives, navigate challenges and communicate the value of partner input. Conclusion By using collaborative decision‐making in the early stages and throughout the study, researchers built trust with partners so that when decisions were difficult to resolve, partners still felt listened to and that their input mattered. Patient or Public Contribution Members of the PCORI Patient Engagement Advisory Panel in 2019–2020 provided input into the design of the study, including the research questions and approaches to data collection and analysis.


| INTRODUCTION
Emerging literature documents the benefits of patient and other stakeholder engagement in research, including active influence by stakeholder partners on study protocols and study enrolment rates. [1][2][3][4][5] These contributions have resulted in impacts on studies' acceptability, feasibility, rigour, relevance and even on the scope and quality of engagement. [6][7][8][9] Study teams may encounter challenges with engagement, such as building and maintaining relationships between researchers and partners, including group dynamics. 10,11 Effective collaboration on multistakeholder teams requires encouraging diverse perspectives while negotiating disagreements and conflict. [12][13][14][15][16] Yet, researchers have voiced challenges in managing conflicting feedback when different perspectives arise during decision-making. 17,18 Further, despite the growing practice and science of engagement, much of the understanding of practices that support negotiating different viewpoints in decision-making has been based on publications that include narratives from single studies or studies with nominal information about such practices. 19,20 To advance our understanding of ways to manage diverse perspectives, we gathered and analyzed information within and across six case studies. Each case was a study team funded by the Patient-Centered Outcomes Research Institute (PCORI).
PCORI requires awardees to engage diverse stakeholders in comparative effectiveness research studies. PCORI provides broad guidance about how to engage partners, 21 which allows study teams to explore practices and address challenges with their available resources and specific to their institutional and project contexts. 4,6,22 The research questions were: • How do study teams manage and balance different viewpoints and priorities when making decisions about study conduct?
• How do engagement practices-such as behaviours, interactions or structures-help or hinder the management of different viewpoints and priorities?

| METHODS
We applied a multiple-embedded descriptive case study design. 23 This approach includes multiple units of analysis in each case (e.g., interviews with researchers and partners, document review) and improves validity through data source triangulation. 24 We included six cases to compare and corroborate findings across cases.
PCORI's Patient Engagement Advisory Panel provided input to the design of the study, including the research questions and approaches to data collection and analysis. The American Institutes of Research Institutional Review Board reviewed this study.

| Sample
We selected candidate cases from a purposeful sample of 58 PCORI-funded projects that participated in a recent qualitative study on the influence and impact of engagement. 7 The original study sample included 110 researchers and stakeholder partners from projects that were a heterogeneous representation of PCORI's portfolio regarding study completion status (i.e., complete and MAURER ET AL. | 1607 active), funding announcement category, PCORI priority content area, study design, study populations and health conditions.
Of the 58 projects, we excluded 16 where interviewees demonstrated difficulty with recall. We then identified 26 cases in which an interviewee described one or more of the following, indicating a difference in viewpoints or priorities: • Deliberation between stakeholder values and scientific or technical requirements.
• Change in engagement structures over time.
• Disagreements or conflicts about a study decision or task.
For eligible cases, we created a blinded list of characteristics, including the rationale, level of engagement experience reported by the PI, types of stakeholder partners involved and project completion date. Five team members independently selected 12 cases for inclusion with the goal of achieving diversity across characteristics.
Combining these lists resulted in 14 cases; the team identified 6 priority and 8 alternate cases.

| Outreach and recruitment
We conducted email outreach to recruit PIs from identified cases between May 2020 and March 2021. Due to the onset of the COVID-19 pandemic, we offered flexible scheduling. We emailed the PI to assess interest, even if immediate participation was not possible. We then scheduled a call to discuss the case study approach, the process for selecting interview participants, how PCORI will use the case study results and concerns about confidentiality or compensation.
Projects were enroled as a case study if the PI was willing to participate in an interview and follow-up activities such as reviewing case narratives, and providing contact information for up to five study team members (researchers, partners) to participate in interviews. After obtaining a list of recommended participants from the PI, we contacted those individuals via email or phone to assess their interest and availability.

| Data analysis
For within-case analysis, analysts first extracted data from individual interviews into a structured memo with standard embedded case study topics, including project characteristics and background information (e.g., number and type of stakeholders, engagement structures), how projects managed and balanced different viewpoints and priorities and how engagement practices helped or hindered balancing or managing different viewpoints and priorities. For five cases, the same analyst (K. The entire team discussed initial findings and determined areas for further examination. We then discussed and iteratively synthesized findings in three areas: different viewpoints and priorities encountered, decision-making processes and engagement structures and behaviours.
Throughout the study, the team met weekly to ensure a consistent approach across cases, discuss emerging findings, share personal reflections on how the data aligned with or differed from expectations and test assumptions, biases and conclusions.

| FINDINGS
We reached out to 11 PIs: 6 agreed to participate, 2 declined, and 3 did not respond to initial or follow-up email requests. Across the six cases, we conducted 28 interviews with 32 interviewees, including 6 PIs, 13 other researchers and 13 partners. Partners included patients, caregivers, parents, clinicians and a healthcare administrator. We did not collect demographic information for interview participants. Table 1 provides an overview of the final six cases.
Below, we summarize each case and present findings from the cross-case analysis.  decision-making about the study conduct. They also wanted more hands-on responsibility for the engagement itself and more ways to engage with the team instead of reading materials on the online portal. The team pivoted to a new design that placed stakeholders in charge of their own group by bringing on one patient and one family caregiver from the core research team as co-chairs of the SAG, which created formal linkages between the research team and the stakeholder group. The team also invited the SAG to attend one research team meeting each month, distributed a monthly newsletter with SAG member spotlights, hosted more inclusive annual in-person meetings and administered experience surveys to identify ways to improve engagement. This new structure created abundant opportunities for stakeholders and researchers to exchange views and shape decisions that took different perspectives into account.

| Within-case analysis
Formal and informal feedback loops also allowed the research team to explain when and why decisions did or did not integrate stakeholder viewpoints.

| Case 2: Building bridges
A gradual process of relationship building resulted in strong partnerships between researchers, clinic leadership and parents. Before the study began, the PI spent 6 months developing relationships with partners, starting with a local clinic that served patients whose first language was Spanish. The PI first invited the clinic's director to a meal and a 'friendly chat'-an informal conversation to exchange ideas and explore common ground-at a restaurant near the clinic.
Before becoming involved in this study, the clinic 'felt burned out' by  actively participating (e.g., sleeping, using their phones, or otherwise distracted). They found that patients were overwhelmingly more active in peer-led education classes and more socially engaged with their peers than they were with nurse educators.
After witnessing its success, the nurse partners became one of the intervention's biggest advocates.
In a couple of instances when interviewees reported major conflict-either resolved or unresolved, it initially resulted in discontent [1] or hurt feelings [5] amongst partners. In case 1, researchers acknowledged the conflict and changed the engagement structure to expand opportunities for partners to contribute. In case 5, the lead nurse partner expressed concerns that the group had not treated them as valuable or important to decision-making, and a researcher apologized for the hurt feelings [5]. A researcher said: It wasn't like somebody was the bad guy here.
Everybody had the same goal… to provide the best care to the patient. That was always front and center.
When that happens and then you start liking each Although all cases described including stakeholder partner input in decisions generally, three cases reported a specific decision where stakeholder input was not incorporated or that appeared to be nonnegotiable [2,5,6]. These decisions focused on study design rigour [2,5] or fidelity to an evidence-based intervention [6]. Similar to case 3, parent partners in case 2 had wanted all patients to access the intervention. However, without a viable alternative, the PI did not want to change the study design from a randomized controlled trial to maintain scientific rigour. The researchers explained why randomization would provide the strongest evidence for the intervention, and the partners understood this reasoning. Also, because researchers had built trust with partners over time and had previously incorporated partner input into decisions, partners still felt heard and valued.

| Engagement structures and behaviours
No one engagement structure emerged as better able to help or hinder management of different viewpoints. All but one case [6] used multiple types of engagement structures (e.g., co-investigator, advisory group, working groups/committees), and all cases also worked with single-or multistakeholder advisory groups. All cases had stakeholder partners in roles with decision-making authority, either as co-investigators, on the core research team, or as part of working groups. T A B L E 2 Engagement behaviours related to managing different viewpoints and priorities.
Category Behaviours (number of cases reported)

Inclusion
Creating meaningful opportunities for partners to contribute • Included partners in decision-making processes and on leadership teams (6) • Worked to integrate multiple perspectives in decisions (5) • Encouraged stakeholders to take active roles in the study (2) • Invited partners to participate in working groups to ensure all perspectives were represented (2) • Shifted engagement structure to be more inclusive (2) • Crafted specific questions for stakeholder advisory group meetings to get focused input (2) • Used a gradual process of conversations to build trust and define the project in a way that is mutually beneficial (1) • Invited stakeholders to research team calls (1) • Provided childcare during meetings (1) • Scheduled meetings at a time and location convenient to stakeholders (1) Eliciting and synthesizing diverse viewpoints • Remained open to change (6) • Listened and responded to stakeholder perspectives (6) • Had members of the research team who served as cultural bridges or liaisons (2) • Set expectations for stakeholder participation (2) • Sent meeting agenda to stakeholder advisory group members in advance (2) • Used first names to diminish hierarchy (1) • Set tone by setting and repeating the ground rule that we are all equal on the team (1) • Focused on what was best for the project rather than individual perspectives (1) • Developed a shared mental model to get everyone on the same page during study objectives (1)

Making interactions informal and fun
• Had meetings in a comfortable space and provided food (2) • Participated in rituals of celebration, such as life events, birthdays or holidays (2) • Included time for informal conversations during meetings (2) • Held retreat at a family-friendly resort (1) • Planned informal dinners (1) • Used icebreakers to increase comfort (1)

Open and intentional communication
• Had feedback loops from researchers to partners about how input was applied (5) • Maintained open communication (4) • Conducted one-one-one outreach to stakeholders (3) • Reported being transparent about study decisions and activities (2) • Acknowledged or apologized for unintended harm, such as discontent or hurt feelings (2) • Shifted communication approach to be more inclusive, which included distributing a newsletter and designating a point person (1) Before managing different viewpoints and priorities, teams had to first invite and elicit diverse viewpoints. These inclusion behaviours related to both creating opportunities for partners as well as eliciting and synthesizing diverse viewpoints in ways that worked for their team and study. Summarizing these types of behaviours, one partner said: They always asked our opinion. Always. Even at the very end when they were writing up the paper, they were like 'hey, who wants to be a part of this, who wants to read it?' They really wanted us to be involved, and that was obvious because they were always asking us to be involved. (Case 3, Stakeholder) In a respectful and welcoming environment, partners felt that they had valuable expertise to share. These behaviours focused on setting a respectful and collaborative tone, such as remaining open to change and listening and responding to stakeholder perspectives.
As one partner noted: The way they respected us, the way they listened to however, cases did not report agreeing to this preset course of action before the decision.
• Decentralized decision-making, or delegating or assigning decisions to team members. Working groups with authority to make decisions for a specific task like recruitment is an example of this type of decision-making model.
In addition to practices used by these cases to facilitate decisionmaking (e.g., having a ground rule, behaviours to elicit and synthesize diverse viewpoints), study teams may consider additional practices to strengthen and improve the transparency of collaborative decisionmaking processes. For example, as identified in other studies, at the start of working relationships, teams may want to work together to clarify roles, determine decision-making approaches, define consensus and agree on a preset course of action if consensus cannot be reached. 18,31,32 Perhaps most important, cases managed different viewpoints to meet the needs of their partners and the study, reflecting other literature that reinforces there is not a 'one size fits all' approach to engagement practices. 18,33 For example, in three cases, patient partners initially objected to randomization when an intervention perceived to be effective would not be available to all study participants. This finding reiterates other published literature, where possible study participants' willingness to undergo randomization drops as they learn about accumulating evidence of a treatment's effectiveness. 34 In two cases, study teams identified alternate study design options that met researchers' criteria for rigour; in the third case, researchers and partners had a productive discussion about the importance of randomization, and partners agreed to continue with the randomized design.

| Limitations and future research
The purposeful sampling approach of PCORI-funded projects may limit the generalizability of the findings. The cases reflected a subset of projects from the original study on engagement's influence and impact, in which managing different viewpoints and priorities emerged as a challenge. Because we did not ask explicitly about these issues in the original study, other projects may have had similar experiences but did not disclose them. Also, although not a selection criterion, all cases in this study used collaborative decision-making. Findings may not reflect projects using other decision-making approaches or where collaborative approaches were not effective.
Further, for four of the six cases, more than 2 years had passed between these interviews and project completion. Interviewees may have not remembered all the strategies and behaviours used to manage different viewpoints and priorities. To enhance recall, interviewers prompted participants to remember specific events described in study progress reports or previous interviews. Finally, we did not conduct interviews with all partners involved in each project; findings may not represent all perspectives about what worked or did not work to balance viewpoints and priorities.
Even with these limitations, these findings provide a foundation for understanding how study teams manage and balance different viewpoints and priorities. Future research could explore how different decision-making models affect researchers' and partners' perceived quality of engagement and decisions about study design and conduct. Studies could also examine relationships between decision-making approaches and the ability to identify and work with partners from historically excluded and marginalized communities.