Long‐term views on chronic kidney disease research priorities among stakeholders engaged in a priority‐setting partnership: A qualitative study

Abstract Background Patients and stakeholders are increasingly engaging in health research to help address evidence‐practice gaps and improve health‐care delivery. We previously engaged patients, caregivers, health‐care providers (HCPs) and policymakers in identifying priorities for chronic kidney disease (CKD) research. Objective We aimed to explore participants' views on the research priorities and prioritization process 2 years after the exercise took place. Design In this qualitative descriptive study, individual interviews were conducted and analysed using an inductive, thematic analysis approach. Setting/participants Participants resided across Canada. We purposively sampled across stakeholder groups (CKD patients, caregivers, HCPs and policymakers) and types of engagement (wiki, workshop and/or steering committee) from the previous CKD priority‐setting project. Results Across 23 interviews, participants discussed their research priorities over time, views on the prioritization process and perceived applicability of the priorities. Even though their individual priorities may have changed, participants remained in agreement overall with the previously identified priorities, and some perceived a distinction between patient and HCP priorities. They tended to balance individual priorities with their broader potential impact and viewed the prioritization process as systematic, collaborative and legitimate. However, participants acknowledged challenges to applying the priorities and emphasized the importance of communicating the project's outcomes upon its completion. Conclusion Two years after engaging in CKD research prioritization, stakeholder participants remained in agreement with the previously identified priorities, which they felt reflected group deliberation and consensus. Rapport and communication were highlighted as key elements supporting effective engagement in research prioritization.


| INTRODUC TI ON
Chronic kidney disease (CKD), characterized by persistently abnormal kidney function, affects up to 13% of the population globally 1 and is associated with considerable morbidity, mortality and healthcare costs. 2 However, there is a lack of high-quality evidence to guide CKD treatment, 3,4 and even where evidence exists, it is not always translated into practice, thereby leading to evidence-care gaps. 5 One potential way to address such gaps and optimize delivery of patient-centred CKD care is to engage research end-users (ie patients, caregivers, clinicians and policymakers) in establishing the research agenda. 6,7 Because the priorities of those most impacted by research findings may not align with those of the research community, redirecting the focus of research to stakeholder-identified priorities may enhance the relevance and uptake of evidence into practice. 6,8,9 In 2015, we used the approach established by the James Lind Alliance (JLA) to identify the most important, unanswered questions about CKD management among CKD stakeholders, including adult patients with non-dialysis CKD, their caregivers, and health-care providers (HCPs) and policymakers involved in CKD care. 10, 11 The final step of this internationally recognized process includes a 1-day workshop where participants convene to discuss and rank the top 10 research priorities. To address the limitations of an in-person meeting, such as travel and resource requirements, we conducted a randomized controlled trial comparing the traditional in-person workshop to a novel, online wiki-like platform for ranking the final priorities. 12 Although there was some overlap between the two groups' top 10 priorities, findings from a post-intervention survey indicated that wiki participants felt they were less able to express their views and contribute meaningfully. 12 The priorities identified through this exercise, as well as those from a related JLA partnership for those with advanced CKD on or nearing dialysis, were used to inform all research projects within a national patient-oriented kidney research programme in Canada. 13 The impact of patient and stakeholder engagement on participants and research has not been fully assessed, 14,15 and existing reports have tended to focus on short-term perceived impacts of engagement. 16 Of those studies involving patients in research prioritization, none has examined participants' views on the priorities beyond the completion of the priority-setting exercise. As research programmes and funding decisions increasingly address priorities identified through processes such as that of the JLA, [17][18][19] it is important to understand how stakeholders who contributed to the research priorities consider them over the long term. In this qualitative study, we sought to characterize how stakeholder participants from a CKD priority-setting project viewed the research priorities and the prioritization process 2 years after the exercise took place.

| Study design and setting
This study was guided by a qualitative descriptive methodology, 20,21 which allowed us to explore participants' experiences with the priority-setting project and their individual approaches to determining CKD priorities. The full JLA process for this project has been described previously. 10,12 Participants included CKD stakeholders from across Canada. The steering committee (n = 13) met regularly over a 1-year period to determine a priorities' shortlist, from which the final 10 priorities were ranked. In the final prioritization step, we compared a traditional, 1-day, in-person workshop held in Toronto, Canada, in June 2015 (n = 26) to a 2-week, online, wiki-like process (n = 27) for group ranking of the priorities. 12 Eleven of the steering committee members attended the in-person workshop (as participants or facilitators).
We observed principles of rigour in the design and conduct of this qualitative study, including suitability of the research question to qualitative research, transparency in our design and sampling, provision of sufficient support for our findings and clear integration and interpretation of the data. 22

| Participant selection
For eligibility in the original priority-setting project, English-speaking, adult participants from a relevant CKD stakeholder group had to have had access to high-speed Internet and high health literacy 23 to meet the technical requirements of engaging collaboratively online. Fifty-three participants from the original project were eligible for inclusion in this study (Appendix S1). Participants had taken part in one or more aspects of the project (steering committee, workshop or wiki) as a CKD stakeholder (patients with non-dialysis CKD, caregivers, HCPs and policymakers). To identify unique perspectives and common patterns across varied roles in the project, we purposively sampled among all stakeholder groups and types of engagement. 20,24 Eligible participants were invited by email to participate in a telephone interview; those who resided in Toronto were given the option of a face-to-face interview. The use of both telephone and inperson interviews in the same study yields similarly acceptable data and should not compromise the quality of the findings. 25

| Data collection
We conducted in-depth, semi-structured interviews to explore participants' views on CKD research priorities and experiences with research prioritization. All interviews were conducted by a single interviewer (M.J.E.). One week prior to their interview, participants were provided with the original shortlist of 30 CKD research priorities and the top 10 ranked priorities from the workshop and wiki groups. Participants were asked to reflect on these priorities and consider how they might rank them now. We referred to an analytic-deliberative conceptual model for stakeholder engagement in research 26 when developing our interview guide, which addressed their experiences with the priority-setting project and their previous and current priorities for CKD research (Appendix S2). We pilottested the interview guide with a qualitative researcher experienced in patient engagement and made minor revisions to it following each of the first three interviews (ie to the wording and ordering of questions, not the content of the interview guide). All interviews were audio-recorded, transcribed verbatim and uploaded in NVivo 11 (QSR International Pty Ltd, Victoria, Australia) to facilitate data organization and coding. Demographic information (ie age, sex, province of residence, kidney function [patients/caregivers] and job title [HCPs/policymakers]) was collected for the purposes of summarizing our sample and contextualizing our findings.

| Data analysis
Data collection and analysis took place concurrently. Guided by a thematic analysis approach, 27 two researchers (M.J.E. and Z.G.) independently generated initial codes representing each expressed idea to systematically organize the data within and across transcripts. The researchers met after coding the first three transcripts to discuss the evolving coding scheme and again after coding every three to four subsequent transcripts to refine this scheme and discuss analytic insights. Three other members of the research team reviewed the first three transcripts to ensure relevant data were captured and offer preliminary insights. No new codes were identified beyond the first 10 transcripts, and subsequently, only minor adjustments to code definitions were made. We organized coded data extracts relevant to the research question into preliminary themes, which we then reviewed and refined to ensure clear connections among them and with the coded data. Developing findings were discussed among the larger research team, and direct quotes were highlighted to illustrate our findings. Data saturation was achieved when no additional relevant data were collected in the interviews. In this report, our use of the term "stakeholders" broadly refers to all stakeholder roles included in this study (ie patients, caregivers, HCPs and policymakers), unless otherwise specified.

| RE SULTS
Of the 53 eligible participants, 23 completed an interview (20 by telephone; 3 face to face) lasting approximately 1 hour. Of those who did not participate, 5 people declined, 4 could not be reached, and 21 did not respond to our email invitation. As we achieved data saturation and representation across stakeholder groups, we did not make further attempts at recruitment. Of the 23 participants,

| Research prioritization process
When weighing the relative importance of the CKD research pri- There was a lot of give and take, and you could've easily taken out three or four or five and put in three or four or five different ones, but it would've just been as acceptable. It was hard narrowing it down to 10, that's for sure. (ID7, caregiver) Despite these challenges, the process for identifying and ranking CKD research priorities was perceived as systematic and rigorous.
One HCP said that she felt the participants "knew exactly what they needed to do at every stage and they knew how to effectively participate" (ID20), and for one policymaker, that this multi-stakeholder approach to research prioritization "does seem like a pretty solid way of moving forward" (ID9).
Across all interviews, the in-person workshop was considered the preferred method for identifying research priorities. For example, one workshop patient speculated that there "was a much better rapport" (ID10) in person, and another questioned whether or not "you would get that type of interaction in an online forum" (ID5). The three HCP participants who had experience with both in-person and wiki formats for research prioritization all expressed a preference for the in-person format. In general, wiki participants appeared to be discouraged by the low participation rates and lack of justification for rankings through the online chat feature. One wiki participant related the lower engagement in this format to the lack of opportunity for team building and rapport: The discretionary decision to participate or not is based on a commitment that involves relationships. Some participants acknowledged potential advantages to the wiki, such as convenience and comfort expressing one's opinion. Further, many participants remarked upon the similarity between the final workshop and wiki top 10 lists. 12 One patient, who expressed being sceptical initially, subsequently suggested the overlap in the two groups' priorities "proves the concept that there is some validity to this wiki stuff" (ID12). In contrast, one policymaker regarded the failure to communicate a project's impact as a "silo symptom" (ID11

| D ISCUSS I ON
In this study, we found that stakeholders from a CKD research priority-setting exercise remained in agreement with the priorities The final step in the JLA approach is rooted in deliberative methods whereby participants are encouraged to discuss their views, consider different perspectives and arrive at a reasoned group decision (ie final top 10 priorities). 11,33 Participants in our study appeared to view this process as systematic, inclusive and equitable through all of its stages; they perceived their interactions as respectful, denied tensions between stakeholder groups and suggested that consensus was achieved. However, participants expressed a preference for the in-person format over the online wiki-based platform for the final prioritization step due to perceived barriers to communication and rapport online. The views of participants in this study endorse identified barriers to online collaborative writing application use in health-care settings, such as tool unfamiliarity, time constraints, technical concerns and frustration by low participation. 34 Familiarity and relationships among group members have been identified as an important element underlying effective stakeholder engagement, 35 and in this setting, it could be strengthened by allowing participants to meet informally beforehand (eg via teleconference), as one of our study's participants suggested.
Prioritization exercises such as that of the JLA are being used to identify and prioritize research topics on an international scale and across a number of health conditions. To date, more than 60 JLA priority-setting partnerships have identified priorities for conditions such as asthma, schizophrenia and urinary incontinence, among others, which have been used to inform research programmes worldwide. 11 The JLA is making a concerted effort to track the impact of published priorities resulting from its priority-setting partnerships.
However, this can be challenging in the light of the public availability of the priorities and the fact that researchers can interpret and apply them as they choose. 18  (Can-SOLVE CKD) Network. 13 The research activities within this network were derived in large part from patient and stakeholder priorities identified through JLA partnerships across the CKD spectrum.
Although participants in our study seemed to agree on the importance of the final priorities, some raised concerns about their scope, clarity and feasibility. To some HCPs, the research priorities seemed too vaguely worded and/or broad in scope to permit their operation- Finding this balance between generality and specificity in defining the priorities is imperative to facilitating their subsequent application as research questions. 36 The limitations of this study relate primarily to issues of study design and sampling. First, as 2 years had elapsed since the CKD priority-setting project, participants may not have recalled details about the prioritization processes or their individual priorities.
However, we focused on participants' long-term views and subsequent experiences, and we provided participants the final priorities' lists in advance of the interviews to prompt discussion. Second, we acknowledge that in this study, participants discussed their individual approaches to research prioritization, whereas prioritization at the time of the priority-setting partnership occurred through group deliberation and consensus. We also did not collect individually ranked priorities at the time of the workshop/wiki. However, elicitation of participants' current priorities was not intended for comparative purposes, but rather to encourage reflection and stimulate discussion on why and how they prioritize certain areas over others. Third, participants were heterogeneous in that they were from different stakeholder groups and had participated in different aspects of the project, including some who took part in more than one step (eg some steering committee members also participated in the workshop). As we aimed to capture both the diversity in perspectives and common patterns across this variation, we intentionally sampled across these parameters and feel this contributed to the collection of rich data. Despite the variation in type and extent of engagement, participants expressed a perceived validity of the JLA priority-setting process overall and shared the view that engagement through the online wiki-like platform did not allow for the same sense of partnership and familiarity as direct, interpersonal contact.
Lastly, we acknowledge that our findings may be context specific and that participants' priorities and experiences with research prioritiza- decisions remained valid and reflective of the deliberative process. However, the implications of the resulting priorities must be carefully considered in relation to those who were included in the prioritization process. The fact that certain populations may be underrepresented in such exercises (eg those with advanced illness or from marginalized groups) highlights the need to adopt inclusive strategies for engagement that ensure balanced representation of all relevant perspectives. 37 Further, most participants in our study were unaware of the outcomes of the CKD priority-setting project, thus raising questions about how to optimize stakeholder engagement and expectations for follow-up upon study completion.
In a case study of a UK Research and Development consortium, the most important concern raised by public volunteers was their desire for feedback on the value of their contributions and the research outcomes. 38 Others considering future research prioritization partnerships should define strategies at the outset for dissemination of the priorities and their impact to participants. Lastly, participants' experiences with different prioritization formats (ie in person vs online) highlight the need for future study on alternative approaches to research engagement, particularly for those with health limitations who may not otherwise be able to participate. An acceptable format must not only permit the research task at hand but also facilitate rapport and communication among participants.

| CON CLUS ION
Two years after engaging in a CKD research prioritization exercise, stakeholders remained in agreement overall with the previously identified priorities. Even though their individual experiences may have influenced their views on the priorities, participants suggested that group deliberation and consensus were key elements supporting the prioritization process and credibility of identified priorities.
Participants emphasized communication, feedback and rapport among participants and the research team, thus highlighting opportunities for future work to enhance the experience of stakeholders collaborating in research.

ACK N OWLED G EM ENTS
We would like to thank Ms. Christine Marquez for helping develop and pilot-test the interview guide. We would also like to extend our sincere thanks to the participants of this study, who generously gave their time to take part in both the original priority-setting exercise and subsequent interviews to help us better understand their experiences.

CO N FLI C T O F I NTE R E S T
The authors have no conflict of interests to disclose.