Experiences and lessons learned from a patient‐engagement service established by a national research consortium in the U.S. Veterans Health Administration

Abstract Introduction Meaningful engagement of patients in the research process has increased over the past 20 years. Few accounts are available of engagement infrastructure and processes used by large research organizations. The Pain/Opioid Consortium of Research (Consortium) is a U.S. Department of Veterans Affairs (VA) research network that provides infrastructure to accelerate health research and implementation of evidence‐based health care. The Consortium's key activities include facilitating Veteran‐engaged research and building community between Veterans and VA researchers. This report sought to describe experiences and lessons learned from the first 3 years of a national research engagement service, featuring a Veteran Engagement (VE) Panel, established by the Consortium. Methods We gathered authors' experiences to describe development and operation of the Consortium's VE Panel. Engagement staff collected program evaluation data about partners (Veterans and researchers), projects about which the VE Panel consulted, and meeting attendance during operation of the engagement service. Results We created a 12‐member VE Panel; all of whom had lived experience with chronic pain, prescription opioid medication use, or opioid use disorder. Engagement staff and VE Panel members implemented an engagement service operational model designed to continuously learn and adapt. The panel consulted on 48 projects spanning the research process. Seventy‐eight percent of panel members, on average, attended each monthly meeting. VE Panel members and participating researchers reported high satisfaction with the quality, ease, and outcomes of their engagement service experiences. Conclusions This work provides an illustrative example of how a national research consortium facilitated Veteran‐engaged research and built community between Veterans and VA researchers by developing and operating an ongoing engagement consulting service, featuring a VE Panel. The service, designed as a learning community, relied on skilled engagement staff to cultivate high quality experiences and outcomes for all partners.

community, relied on skilled engagement staff to cultivate high quality experiences and outcomes for all partners.

K E Y W O R D S
engagement infrastructure, patient engagement, patient-centered research, research networks, Veterans

| INTRODUCTION
Meaningful engagement of patients and caregivers in research is a critical component of learning health systems. 1,29][10][11] Funders have encouraged or required patient engagement in health research. 12,13In the United States., the Patient-Centered Outcomes Research Institute (PCORI), 14 requires patient engagement throughout studies, 7 manages a research portfolio on patient and stakeholder engagement, 15 and offers training and resources. 16[19] As the largest integrated learning health care system in the United States, VA funds research on health issues that affect Veterans. 20Increasingly, VA HSR leadership seeks to include Veterans in all steps of the research cycle. 20In 2015, VA HSR began supporting Veteran engagement in research by establishing a national Veteran Engagement (VE) Workgroup to explore approaches and develop recommendations for promoting engagement. 21HSR intramural funding now requires consideration of Veteran engagement in research proposals. 22[25][26] Sharing of models and lessons learned may support successful implementation of patient engagement efforts in VA and beyond; however, few accounts are available of engagement infrastructure and processes used by large research organizations. 19,27,28is article describes experiences and lessons learned from the first 3 years of an engagement service established by a national VA research consortium to facilitate Veteran-engaged research and build community among Veterans and VA researchers.We describe the service's context, activities, output, and evaluation results.We also share templates and tools to adapt and use in other settings to promote meaningful patient engagement.

| FRAMEWORK AND METHODS
We adapted the PCORI engagement in research theory of action model 29,30 to the circumstances of the Consortium's engagement service (Figure 1) to use as a framework to describe our experiences.We use the term "engagement service" to reflect the Consortium infrastructure and processes involved with creating and operating a standing engagement panel as a resource available to researchers.The PCORI model frames engagement activities, their quality, and the resulting research and partner outcomes within a given context made up of resources and circumstances that may influence how engagement occurs and its impact.We added an output domain to capture concepts of productivity and participation, adjusted descriptions to a VA setting, and included only shorter-term research outcomes to align with our scope of evaluation.
We employed multiple methods of data collection including authors' experiences, surveys, and interviews (Table 1).Our approach, in keeping with the principle of partnership in research engagement, involved staff, Veterans, and researchers to characterize the nature and outcomes of the Consortium's engagement service.Authors TS, AJ, EA, MA, and MM are engagement service staff, GT is a stakeholder engagement consultant, EK is a physician-researcher, and RK is a VE Panel member.

| VA research setting
In 2022, investigators at 104 VA medical centers across the country conducted research spanning the translational spectrum, with a total research budget of $2.3 billion. 31VA HSR funds collaborative research to identify, evaluate, and rapidly implement evidencebased strategies that improve care quality and safety. 32HSR intramural funding requires consideration of Veteran engagement in research proposals. 22Although the full scope of Veteran engagement activities and practices across VA Research has not been reported, engagement infrastructure to facilitate Veteran involvement in research is documented in individual studies, 33,34 a research resource center for engaging rural Veterans, 35 and at HSR's 20 research centers. 9,10Investigators face common barriers of time, funding, and training to meaningfully involve Veterans in the research process. 36,37 I G U R E 1 VA Pain/Opioid Consortium's engagement service framework.
T A B L E 1 Engagement service framework domains, concepts, and data sources.

| Engagement service operation
Beginning August 2020, staff and panel members provided active engagement services to VA researchers.Staff facilitated three phases of operations: researcher recruitment and scheduling; meeting processes; and continuous improvement (Figure 3).

| Continuous improvement
We established engagement service practices to monitor VE Panel processes, track output, evaluate process and outcomes, and continually apply the knowledge gained.Other characteristics tracked in the spreadsheet included Veteran attendance and speaking order, researcher location, project title, and funding status (Appendix S6).

Post-meeting evaluation
We collected short-term process and outcome evaluation data via online surveys of Veterans and researchers.For Veterans, we developed a panel member survey based on an existing HSR Veteran Engagement Toolkit. 43Questions assessing experience with effectiveness, ease of participation, and sense of partnership in the engagement consultation were emailed immediately following each meeting

| Templates and tools
The appendices contain templates and tools others can adapt and use to create and operate an ongoing engagement service for a research network or enterprise.Table 4 lists materials included.defined in Table 1.

| Engagement output: productivity and participation
Between    These observations are consistent with others' findings in the context of research studies, 12,[45][46][47] local research centers, 10,23 and research networks. 27Our experience suggests that these elements also contributed to another measure of success for ongoing engagement panels: sustained participation by panel members over time.
Participating researchers' reflections indicate that participation in the Consortium's VE Panel meeting process acted as a form of experiential learning to enhance their capacity to meaningfully engage Veterans in future research.Other teams that also rely on skilled staff reported similar findings. 10,28,48Our experience adds to the growing body of evidence 10,48 that durable patient engagement infrastructure can provide researcher capacity-building benefits in addition to addressing time and resource constraints, commonly reported by researchers as barriers to meaningful patient engagement in research processes. 36,47,49,50 conceptualized the Consortium's engagement service as a learning community of VE Panel members, staff, and researchers.This

Figure 2 .
Two Veterans resigned from the VE Panel in 2022 and were replaced by new members in March 2023.Panel member orientation An orientation was created to develop strong group bonds, a deep understanding of the VE Panel member role and responsibilities, and confidence in members' capacity to meaningfully contribute to group activities.Part 1 (1 h) featured introductions, technology review, virtual meeting tips, confidentiality and membership agreements, and panel member payment processes.Part 2 (2 h) included group icebreaker activities; overview of research, the Consortium, and how input would be used by researchers; a discussion of privacy and confidentiality expectations; and a post-meeting evaluation.Part 3 (2 h) included additional relationship-building activities, review of member feedback from the previous meeting, and the first consulting session in which the panel provided feedback on recruitment materials for a study.Orientation exercises emphasized that the unique value of member feedback is derived from their lived experience and opinions, not knowledge of research methods or processes.At the request of VE Panel members, staff established a consistent meeting schedule to allow members to plan around other commitments.

1 .
To fill monthly VE Panel meeting slots in the first year, Consortium staff actively marketed the engagement service by email, announcements, and targeted outreach to VA pain/opioid researchers.After 1 year, word-of-mouth and routine promotion in the Consortium's newsletter were sufficient to recruit researcher clients to the engagement service.Interested VA researchers contacted the Consortium's VE staff.To decrease time spent explaining the VE Panel meeting process, staff developed a one-page overview of the meeting process to share with prospective researchers (Appendix S3).2.2.6 | Engagement panel meeting processEach cycle of a VE Panel meeting involved four phases: (1) plan, (2) prepare, (3) meet, and (4) closeout and summarize.Three roles (facilitator, co-facilitator/technology monitor, and notetaker) with predefined duties allowed staff to rotate among roles.Plan.Two months before a scheduled VE Panel meeting, a staff member solicited client researcher completion of an intake form about their project and goals for the consultation (Appendix S4) and scheduled a 45-min planning session.During the session, two meeting facilitators oriented the researcher(s) to the Panel's meeting process and identified aspects of their project that could most benefit from Veteran input.2. Prepare.Before each VE Panel meeting, a lead facilitator drafted a facilitation guide, which included a plain language summary of the researcher's study to share with the Panel and open-ended key questions and probes to elicit members' perspectives.The co-facilitator and engagement consultant reviewed and edited draft documents.The lead facilitator and engagement consultant conducted a practice meeting with the researcher(s) several weeks before the VE Panel meeting.This live-editing practice session T A B L E 2 Examples of guiding principles reflected in engagement service activities.

4 .
enabled adjustments to better manage time, improve process, or reprioritize key questions.After the practice session, the lead facilitator made final edits to the meeting materials.Staff mailed hard copies of meeting materials to Panel members 2 weeks in advance and sent electronic copies via email 1 week prior to the meeting.Several days before the meeting, staff sent an email reminder, including meeting materials, to all attendees.Hard copy meeting materials lessened panel members' visual burden by allowing them to focus on virtual attendees rather than meeting material handouts on-screen.3. Meet.During the 2-h VE Panel meeting, the lead and co-facilitator took turns leading the discussion and managing comments provided via the meeting platform's chat function.Meetings were not recorded.A staff notetaker captured Veteran feedback, any electronic poll results, and the chat transcript.The researcher's sole responsibility was to be fully present to listening to and interacting with the VE Panel members, including asking or answering clarifying questions.Meetings started with an icebreaker question to help foster connections among panel members, staff, and the researcher(s).Icebreakers helped make the virtual meetings feel more personal, gave all meeting participants a chance to speak right away, and provided a transition to the discussion topics.Next, facilitators briefly recapped feedback provided by members at the previous VE Panel meeting and how the researchers planned to use it (Appendix S5).This "reporting back" is a best practice in stakeholder engagement, clearly communicating the value of members' input and demonstrating engagement principles of partnership and respect.The main portion of the meeting began with a facilitator sharing the plain language overview of the researcher's project followed by the facilitators leading discussions based on the two to four key questions developed with the researcher.To support discussion, meetings sometimes included features such as visual aids, demonstrations, virtual breakout sessions, and live poll questions.Early in the VE Panel's first year, members indicated a preferred approach for eliciting their feedback during meetings: invite two to three members to verbally respond to each key question rather than a "round-robin" or volunteer format.This provided two benefits: (1) a few members could provide in-depth responses and (2) space was created for less outgoing members to actively participate.This practice was augmented by inviting all other members to also offer their input by adding comments or questions via the meeting platform's chat function.One facilitator monitored the chat function and incorporated specific comments into the meeting dialogue.Staff tracked and rotated panel member speaking order for each discussion to support equitable opportunities to contribute from meeting to meeting.Closeout and summarize.Immediately following each meeting, payments to panel members were processed.Typically, within 1 week after each meeting, staff summarized Veteran feedback shared during meeting discussions, appended a copy of relevant chat comments, and sent these to the researcher(s).

T A B L E 3
Characteristics of Veteran engagement panel members a (N = 12).

F I G U R E 2
Residential locations of Veteran Engagement (VE) Panel members and VA locations of engagement service clients.Engagement service VA client locations are defined by the scope of the consult (i.e., national or local) and, for local clients, the city and state of the VA medical center of the lead researcher.F I G U R E 3 VA Pain/Opioid Consortium's engagement service operational model.Monitor After each VE Panel meeting, staff spent up to 15 min reflecting on outcomes of the meeting and any concerns with group member agreements, guiding principles, or pace.If warranted, reflections were triaged for either immediate follow-up or consideration at a quarterly staff reflection meeting.Engagement service tracking Staff captured characteristics of each VE Panel meeting, researcher, and project in a tracking spreadsheet with a "dashboard" of calculated fields to track engagement service productivity and patterns.We reviewed final VE Panel meeting facilitation guides to verify VE Panel consultation category (i.e., study, governance, implementation), topic (e.g., research priorities, study design), type of study (e.g., clinical trial), and study stage (e.g., study design, recruitment).

(
Appendix S7).After the VE Panel's first year, based on Veteran feedback, we changed our short-term evaluation process to an openended virtual poll question at the conclusion of each meeting ("Please share feedback on today's meeting.What went well, and what is one thing you would like to see improved?").We also collected qualitative process and outcome evaluation data from Veterans, at least annually, during a VE Panel meeting with facilitated discussions dedicated to group maintenance and planning, with optional one-on-one conversations after the meeting with a staff member.To assess experiences of researchers who used the engagement service, we used a survey previously developed by the Minneapolis VA Center for Care Delivery and Outcomes Research Veteran Engagement Core to solicit short-term satisfaction with effectiveness, ease of participation, sense of partnership, learning, and changes in perception or project activities from the engagement consultation (Appendix S8).We also collected follow-up process and outcome evaluation data from researchers approximately 6 months after each VE Panel meeting using semi-structured interviews.During each interview, staff captured notes that were reviewed by a second staff member to corroborate identified outcomes.Questions focused on (1) changes resulting from their consultation with the VE Panel, (2) recommendations of the Consortium's engagement service to colleagues, and (3) free-form comments about experiences with the engagement process (Appendix S9).To close feedback loops, staff reported short-term researcher evaluation results at the next month's VE Panel meeting.Twice per year, staff batched 6-month follow-up researcher process and outcome evaluation results and reported them at a VE Panel meeting.T A B L E 4 List of appendices.month follow-up interview questions T A B L E 5 Characteristics of Veteran Engagement (VE) Panel Consultations, a Sep.2020 to Aug. 2023.Other (e.g., systematic review, standard measure instrument) 3 (10%) a Includes VE Panel meeting consultations (n = 38 across 36 meetings) and other consultations (n = 10).b Study consultations are inclusive of quality improvement or program evaluation projects and may involve one or more study stages as topics.c Veteran presentation venues included in-person, virtual, and recorded.Adapt We established routine learning cycles, which were opportunities for staff and Veterans to reflect on evaluation data and experiences and to adapt engagement service operations to new learning or circumstances.Staff met quarterly to reflect on monitoring, tracking, and evaluation data; changing VE Panel circumstances; and their own experiences.Staff presented recommended operational changes to VE Panel members for consideration prior to implementing them (e.g., altering meeting processes, broadening engagement services to new lines of work, opportunities to build community around VA research).Beginning in 2021, ad hoc consultations in addition to regular VE Panel meetings were scheduled on a case-by-case basis with a subset of VE Panel members.Staff tailored engagement planning and preparation activities for these ad hoc consultations according to individual circumstances.
Descriptive statistics were used to summarize data from the VE Panel tracking spreadsheet, attendance records, and structured evaluation surveys.Two staff members used content analysis44 of project titles to summarize research topic areas covered in VE Panel consultations.All open-ended process and outcome evaluation data were categorized by two staff members into domains of engagement quality, partner outcomes, and research outcomes, as September 2020 and August 2023, the Consortium's VE Panel consulted on 48 different projects with VA clients (Figure 2) on topics spanning the research process from research T A B L E 6 Research topics of engagement service consultations.

T A B L E 8
Engagement quality and outcomes.Evaluation area General examples of quality or change Specific examples or changes attributable to Veteran engagement service Engagement quality Engagement experience • Satisfaction with processes and outcomes • Ease of participation • Co-learning • Sense of partnership "The whole [meeting] process was so incredibly well-thought out and organized.The prep with staff before the meeting made the actual time with the [VE Panel] a truly enjoyable experience… I could really focus on being present to the conversation with the Veterans.The meeting was such a great success in terms of the amount and variety of feedback we received that directly influenced our project."-Researcher "During the pre-meeting prep, engagement staff really helped me shape our questions into a productive structure and get to the points I wanted to get to."-Researcher "I loved the use of the poll to kickstart the discussion… it was interesting to see our commonalities and divergences regarding the poll question."-Veteran "Continue doing what you're doing.The meetings run smooth."-Veteran Partnership functioning and group dynamics
just thought it was really well-done, by both the facilitators and the panel, which was informed, engaged, articulate, and enthusiastic."-Researcher "I felt like a human the whole time.I was neither a disrespected nor elevated human.I got the sense that we were all equals."-Researcher "Keep up the great work … I love being a part of this panel!"-Veteran appreciated the diversity of age, race, gender, and life experiences-that is one of the biggest strengths of this panel."-Researcher "Facilitator did a superb job in moderating the inputs, ensuring each Veteran had time for input, and redirecting lengthy or tangential inputs with sensitivity."-Veteran "Everyone has a voice and is listened to" -Veteran This new professional connection] is a dream come true for me, and it opens doors that I previously thought were out of reach… the support of the VE Panel and engagement staff has instilled in me a renewed sense of purpose and determination and I am immensely grateful for that."- Engagement partners (Veterans, researchers, or staff) emphasized keys to successful experiences and results for partners as: panel member recruitment and compensation, panel orientation, relationship development and maintenance, facilitators addressing power dynamics, and monitoring and evaluating engagement quality and outcomes.