Evaluation of a project to engage patients in the development of a patient‐reported measure for HIV care (the I‐Score Study)

Abstract Background Patient engagement (PE), patients’ meaningful involvement in research through partnerships and sensitivity to their expertise, is receiving attention. However, PE initiatives are poorly reported and little is known about patients’ perspective on PE. Objective To document and evaluate the first phase (22 months) of a PE Project for the I‐Score Study which is developing a patient‐reported measure of HIV treatment adherence barriers, we describe the nature of PE conducted, determine the level of PE achieved and present its impacts from the engaged patients’ perspective. Setting and participants A Montreal‐based committee of ten people with HIV was recruited from community and clinical settings and participated in: I‐Score study decision making, knowledge dissemination, research on the experience of people with HIV and the PE project's evaluation. Methods The evaluation followed a convergent parallel mixed‐methods design. Data collection included participant observation, a satisfaction survey and meeting minutes/transcriptions. Analysis entailed reporting PE activities, generating descriptive statistics and thematically analysing qualitative material. Results PE consisted of twelve meetings, including two focus groups (needs assessment), in addition to four knowledge dissemination activities. PE levels showed an increase: the first four regular meetings entailed information/consultation, while subsequent meetings reached implication/collaboration. Regarding impacts, patients indicated high and stable satisfaction rates (M = 4.4/5; SD = 0.76). Furthermore, thematic analysis identified “positive interactions,” “co‐learning,” “self‐determination,” and “the collective management of confidentiality” as important PE impacts for engaged patients. Conclusion This PE Project evaluation highlighted growing engagement levels, high satisfaction rates and the importance of a patient‐centric approach to PE.


| INTRODUC TI ON
Patient engagement (PE) refers to patients' meaningful involvement in potentially all steps of research to account for their expertise and perspective relative to their health condition, treatments and care. 1 PE represents a shift 2 emphasizing: the importance of values and deliberation in health-related decision making; 3,4 patients' autonomy; 5 sensitive listening and accountability to their circumstances; and partnerships. 6 Given that PE treats patients as actors in research and of their own care, it involves joint action and co-construction of knowledge to empower patients, democratize knowledge and reduce paternalism in health care. [7][8][9][10] Several frameworks 7,9,[11][12][13][14][15][16][17] are available to guide its practice. PE has been conceived as a spectrum ranging from: (a) information (informing patients); (b) consultation (obtaining patients' perspective); (c) implication (accounting for patients' concerns in decision making); and (d) collaboration (partnering with patients in decision making); to (e) empowerment (placing decisions in patients' hands). 16,18,19 PE has broad applicability in research; it is used with different: health conditions (cancer, chronic pain, diabetes, etc.), populations (eg, older people), interventions (eg, physiotherapy, surgery), settings (eg, homelessness, community) and methodologies (eg, systematic review, health technology development). 20,21 It is also gaining greater attention, in part, as it is reported to improve the quality of research and care. 7,8,10,11,22,23 There have been calls since the beginning of the 1980s to involve people living with HIV (PLHIV) in all aspects of the response to the epidemic, 24,25 through PE, among other practices.
Facilitators of PE with PLHIV include: direct communication between engaged PLHIV, care providers, or investigators, 26 active listening to PLHIV's concerns, and emphasis on patient-tailored health-related information PLHIV can use in their daily lives. 27 Challenges noted by UNAIDS 24 and investigators 28 include social and gender inequalities, concerns about disclosure and felt stigma. 24,28 Greater consideration of how engaged PLHIV perceive PE or tackle these challenges is needed. 29 Methodologically, several limitations of PE have been raised.
For instance, it is often unclear what process or model of PE was applied. 10,30,31 PE evaluation designs and methods are generally inadequately described [31][32][33] and many aspects of PE initiatives are underreported, including recruitment, participation rate, patient satisfaction, frequency or details of PE activities, and impacts. 32,34,35 Overall, few studies have incorporated the patient perspective in PE evaluation. [35][36][37][38] Doing so is important as investigators and patients can disagree on patients' functions 35 which may negatively impact patients' retention 39 and satisfaction with PE. 37,40 Furthermore, in Canada, PE uptake remains slow and identifying ways to increase benefits for patients is needed to optimize their enrolment and retention in PE. 33 Involving patients in PE evaluation and gaining their perspective on PE could help. 35 To address these concerns, this paper's objectives are to document, in detail, and evaluate a PE Project's first phase (November 2015-September 2017), reporting: 1) the nature of PE conducted, 2) levels of PE achieved and 3) its impacts from the perspective of engaged PLHIV.

| Patient engagement project context
This article focuses on a PE Project (hereafter, the Project) designed for the I-Score Study, a study launched in January 2016 to create, validate and integrate into HIV clinical practice an electronic patient-reported outcome measure (PROM) of antiretroviral treatment (ART) adherence barriers. Its rationale and exploratory multimethod (qualitative and quantitative) four-step design are explained elsewhere. 41 So far, I-Score investigators have completed its first step, that is, the generation of the PROM's conceptual framework and its items, informed by qualitative interviews with PLHIV, a review of HIV-specific PROMs 42 and a synthesis of qualitative research with PLHIV on ART adherence barriers. 43

| Patient engagement project rationale
The Project was initiated when I-Score Study investigators realized that the PROM's success depended on evidence of its value to PLHIV 44 and other stakeholders.
The I-Score investigators chose a mode of PE consisting of an advisory committee of PLHIV, which promised direct, equitable and continuous engagement of patients in decision making throughout the Study. Such continuous partnerships are reported to optimize research outcomes of PE. 10,31 In addition, investigators understood that combining PE with participation in research could add value to PE. [45][46][47] However, it is important to clearly distinguish PE activities from engaged patients' participation in research. 47

| Design
The evaluation of the PE Project followed a convergent parallel mixed-methods design, 56

| Data collection
Several data collection methods were used, and all Team members were aware of these.
Through participant observation, 60 with DL's work as a facilitator, detailed notes were taken on attendance, meeting duration, changes in facilitators, discussion themes, Team members' functions, and, if applicable, their associated research activity and impacts on Team members.
To identify discussions, after each meeting, DL examined his meeting notes in relation to the topics or tasks proposed in the meeting agenda, adjusting or adding such "themes" as necessary. The meeting minutes, organized by theme, were then transferred to members for validation.
As a part of collecting data on the patient perspective on PE, an anonymized satisfaction survey was used, which was inspired from instruments used in previous engagement initiatives. 61,62 The survey had two sections. A quantitative section allowed members to rate their satisfaction with a 5-point Likert scale on different aspects of the meeting (see Table 2 Each meeting was audio-recorded (qualitative data) to complete observational notes and survey data, and document members' perceived impacts of the Project.

| Data analysis
To describe the nature of PE, we entered meeting characteristics from observational notes into a Microsoft Word table and examined changes over time.
To determine the level of PE achieved, we referred to the International Association of Public Participation's spectrum of engagement, 16,18,19 classifying each Team discussion along the continuum. This was done by considering observational notes, transcripts, tasks given to or determined by the Team members' themselves, member functions and their overall influence in decision making. Attributed level of PE (see Table 3) was validated with the Team and other investigators.
To define impacts of PE from the patient perspective, we generated descriptive statistics for each quantitative satisfaction survey item. We also analysed member-identified impacts by conducting an inductive thematic analysis 63

| Nature of PE
The nature of PE includes details on the meetings' discussion themes, functions held by members and the KDAs.

TA B L E 3 (Continued)
TA B L E 4 Meeting and survey comments exemplifying member-identified impacts of patient engagement in the Project

Positive interactions
Things that went well "Exchanging with others" "The flow of communication" "Good interpersonal contact" "Flow of conversation, people voicing opinions" "Turn to speak" "Flow of conversation, humor" Things members will remember "All participants having their say" Things members learned about themselves "I have to keep on improving the way I receive others' opinions" Co-learning Things members will remember "Elements or factors that can impede regular pill-taking" "The new definition for adherence" "Iatrogenic effects of medication" "'Domino effects.' They are changes that happen from time to time in my life and that have consequences for my adherence to ART, given the uncertainty of the future" Things members learned about themselves "I have a simple experience but it can still help others" "[Good] pressure to put good health practices in place (sport, exercise, supplements)" Things that may impact their practices "To better discuss with my clinician and to get informed without shame" 03: I am realizing I'm crossing a moment of weathering (concept discussed in former meetings) right now, because I think that I'm tired of struggling to take ART.
[spontaneous usage of a concept discussed in a past meeting] 15: We learned a lot of things from the meeting material and from other members. For example, we learned about clinicians' approaches. I thought I had bad luck or that they did not listen to me carefully. When the others spoke about it, it changed, because we noticed and named problems that exist everywhere in clinicians' approaches.

12: I didn't know if there was an interaction between my ART and calcium.
But [06] brought the topic up, about iron, and I asked my clinician. He realized there could be an interaction, and said I have to take [ART] at least two hours after taking the calcium. 15: In the beginning, researchers gave importance to our gender and age. It seemed to be a priority. There are men and women, it is a good thing. And among us members, not everybody has the same problems. By sharing together, we learned from our respective stories and problems, so this diversity should not be neglected.

Self-determination Things that went well
"We chose the name for the group" Things that could be improved "The place of the meeting" "We should begin at 5 PM" "It would be good to have individual lunch boxes" "To improve the presentation of documents, as some charts are unreadable" Things members will remember "The role I play in the Project" "The importance of the committee" 08: Meeting here, close to the place where I met people from [a specific organization] and formed support, buddy systems, makes a difference. I never had to hide here, we can be ourselves, speak openly.
[…] And this room is big and warm and beautiful. Here our thoughts can flow and feel free. 06: We do not want to control or force ideas on researchers, because we do not have this knowledge. We concentrate on what they expect from us: they consult us about our ideas and our experience, and then they see how this fits into their way of doing things. 06: When we looked for our name, how to qualify us, I thought that this exercise clarified many things in what we were doing. We named our expectations, our objectives, and what is engagement for us. It helps us understand. 04: When we wrote the support letter, we thought it was not in our image at first, it did not reflect us. We worked together and we came to an agreement on the language to use. 06: Engagement applies to daily life as well. We were informed, and we do things outside of research when the information may apply. Engagement is part of a broader set of activities for patients. We define from our own situation what is engagement for us. A total of twelve meetings took place during the evaluation period.

Collective management of confidentiality
Meetings 1-2 were devoted to a qualitative needs assessment for the I-Score PROM during which sex-specific focus groups were facilitated by IT and observed by DL. Focus groups lasted about two hours and took place in a room provided by a partner community organization. Subprojects 1 and 4 were introduced to members by investigators, while Sub projects 2 and 3 emerged from members' suggestions (see Table 3). Also, one Team member with a strong academic background became a co-investigator/partner on Subprojects 2,

| Impacts of PE
This section presents the descriptive statistics of the satisfaction survey items and the results of the thematic analysis, offering a patient perspective on PE.  especially with regard to "interest," "relevance," "enjoyment" and the "facilitation."

| Member-identified impacts
The thematic analysis of the qualitative survey items and transcripts generated four member-identified impacts of PE, described below, which help contextualize members' satisfaction ratings. Table 4 presents these themes and illustrative member quotes. This theme is consistent with their high and stable satisfaction ratings, notably on the "interest," "enjoyment" and "meeting with people" items.

Co-learning captures mention of collective learning around health.
In the open-ended satisfaction survey questions, members described learning relevant care-or research-related concepts and claimed to subsequently use them. As mentioned above, ratings on the two quantitative survey items concerned with learning were among the lowest at Meeting 3 but, afterwards, they reached high levels.
According to members, they mostly learned by exchanging Most members also mentioned improving their ability to support other PLHIV (eg, friends, family members) with ART-taking. Some mentioned increased health-related or ART-taking skills (eg, learning to recognize adherence barriers, such as side-effects, and when to discuss these with their providers) and exchanged information on local clinics. A few members claimed to have improved their access to adapted care. Members decided that KDAs were important to transmit lessons learned and ensure the visibility and continuity of the Team, while fighting stigma. However, they agreed that member participation had to be completely voluntary and that KDAs had to be organized for audiences believed to be less likely to stigmatize PLHIV (eg, other PLHIV, health professionals and academics). Members more comfortable with disclosure participated in higher profile activities but expressed concern about "speaking for others."

| Some limitations
The already conceptualized I-Score Study provided the PE Project with initial funding, but there was no PE in its early design. This issue is documented in many PE projects that face insufficient funding or infrastructure to engage patients at early stages, when establishing research priorities and designing projects. 82