The benefits, challenges, and best practice for patient and public involvement in evidence synthesis: A systematic review and thematic synthesis

Abstract Introduction Despite the growing evidence on patient and public involvement (PPI) in health research, little emphasis has been placed on understanding its quality and appropriateness to evidence synthesis (ES) and systematic reviews (SR). This study aimed to synthesise qualitative evidence on the benefits, challenges, and best practices for PPI in ES/SR projects from the perspectives of patients/public and researchers. Methods We searched Ovid MEDLINE, Ovid EMBASE, Cochrane Library and CINAHL Plus. We also searched relevant grey literature and conducted hand‐searching to identify qualitative studies which report the benefits and challenges of PPI in individual ES/SR projects. Studies were independently screened by two reviewers for inclusion and appraised using the Joanna Briggs Institute's Qualitative Tool. Included studies were synthesised narratively using thematic synthesis. Results The literature search retrieved 9923 articles, of which eight studies were included in this review. Five themes on benefits emerged: two from patients'/public's perspective—gaining knowledge, and empowerment; and three from researchers' perspective—enhancing relevance, improving quality, and enhancing dissemination of findings. Six themes on challenges were identified: three from patients'/public's perspective—poor communication, time and low self‐esteem; and three from researchers' perspective—balancing inputs and managing relations, time, and resources and training. Concerning recommendations for best practice, four themes emerged: provision of sufficient time and resources, developing a clear recruitment plan, provision of sufficient training and support, and the need to foster positive working relationships. Conclusion Highlighting the benefits and challenges of PPI in ES/SR projects from different stakeholder perspectives is essential to understand the process and contextual factors and facilitate meaningful PPI in ES/SR projects. Future research should focus on the utilisation of existing frameworks (e.g., Authors and Consumers Together Impacting on eVidencE [ACTIVE] framework) by researchers to help describe and/or report the best approaches and methods for involving patients/public in ES/SRs projects. Patient and Public Contribution This review received great contributions from a recognised PPI partner, the Chair of the Cochrane Consumer Network Executive, to inform the final stage of the review (i.e., interpretation, publication and dissemination of findings). The PPI partner has been included as an author of this review.


| INTRODUCTION
Involving patients/public in research is increasingly regarded as essential to achieving high-quality, relevant, patient-centred research outputs which contribute to improving health and facilitating sustainable health service delivery. 1 Patient and public involvement (PPI) describes the active process of effective partnership working with patients and members of the public to contribute to how health research is planned, managed, designed, conducted and disseminated. 2 It is well-established in high-income countries such as Canada, the United States of America and the United Kingdom, 3 where compelling ethical and political discourses emphasise the rights of patients/public to have a 'voice' in public services, 4,5 as well as the pragmatic considerations and policy response in identifying involvement as a condition for research funding, 6 and the potential for PPI to enhance the relevance of research outputs. 7 The 7Ps framework of stakeholder engagement identifies and categorises various stakeholders involved in health research as follows: providers, policymakers, payers, purchasers, product makers, principal investigators (e.g., researchers) and patients/public. 8 This framework serves as a guide for researchers on which specific stakeholder(s) to involve in research project(s), as well as the stakeholders' roles/responsibilities in these research project(s). The UK's National Institute for Health and Care Research (NIHR) defines patients/public as 'patients, potential patients, carers and people who use health and social care services, people from organisations that represent people who use services'. 9 Notably, patients/public are important stakeholders in research priority setting and funding. 10 However, notwithstanding the positive strides made towards stakeholder involvement in health research, there is still uncertainty about how researchers might best involve patients/public in different types of research and how the impact of their involvement on the conduct and quality of research should be evaluated. 11 Evidence synthesis (ES), also referred to as systematic review (SR), a rigorous and widely used research methodology in public health and medical research, seeks to produce the least biased answer possible to a specific clinical or public health question, by identifying, critically evaluating and synthesising all the relevant evidence on the topic. In recent times, there have been calls for greater inclusion of patients/public and other relevant stakeholders in ES to enhance transparency and improve the relevance of research addressing public and policy health priorities. 12,13 The potential integration of patients/public into evidence-informed health research is vital to enhancing desired patient-centred health outcomes, improving the public's confidence in the healthcare system, and making informed decisions about the care of individual patients. 14 The NIHR's UK Standards for Public Involvement 15 provides a framework for effective public involvement in research in general. It underpins how researchers could use this framework to review their plans for PPI, as well as how research funders could assess the implementation of PPI in research projects in general. The Authors and Consumers Together Impacting on eVidencE (ACTIVE) framework, 16 however, focussed specifically on how to involve diverse stakeholders in SRs. The ACTIVE framework provides a structure that summarises key components of the engagement of knowledge users in SRs, and this is becoming increasingly influential in shaping Cochrane's approach to involvement. Our review builds on the existing ACTIVE framework by enhancing our understanding of best practices for researchers to involve specific stakeholders (i.e., patients/ public) in an ES/SR project, considering the perspectives of both researchers and patients/public.  A study by Cottrell et al. 17 provided evidence on the benefits and challenges of stakeholder engagement in SRs and reported key themes on benefits (such as 'ensuring transparency and accountability', 'establishing credibility', etc.) and challenges (such as 'time', 'finding the right people', etc.). However, the 'benefits' as defined in this study did not capture the personal benefits of involvement, the benefits of training stakeholders about the research process, and the benefits of fostering relationships. Furthermore, whilst this paper explored the benefits and challenges from stakeholders' perspective in general, to the best of our knowledge, there is no qualitative ES reporting on the benefits and challenges of PPI in ES/SRs from the perspectives of specific groups of stakeholders (i.e., researchers and patients/public). The current review, therefore, aims to address this gap in the literature. This paper aims to 1. Synthesise qualitative evidence on the benefits and challenges of PPI in individual ES/SR projects from the perspectives of patients/ public and researchers.
2. Synthesise qualitative evidence on the best practice(s) for researchers to effectively involve patients/public in individual ES/SR projects.

| METHODS
The reporting of this qualitative SR followed the Enhancing Transparency in Reporting the Synthesis of Qualitative Research (ENTREQ) framework. 18

| Operational definition of terms
Due to the conceptual differences and the lack of a consensus on the terms used to describe PPI in research, 19,20 the following operational definitions were used in this review: 1. 'Patients/public' and 'PPI' are defined according to the UK's NIHR's definitions for 'the public' and 'public involvement', respectively. 9 Thus, patients/public refers to 'patients, potential patients, carers and people who use health and social care services, and people from organisations that represent people who use services'. PPI denotes 'research being carried out "with" or "by" members of the public, rather than "to", "about" or "for" them'.
2. 'Benefits' is defined as the potential positive impact/influence on the quality, conduct and dissemination of ES/SR projects and/or the opportunities associated with involving individuals in the synthesis/review process. Thus, describing personal impact on individuals and impact on synthesis/review studies.
3. 'Challenges' is defined as the problems, difficulties or barriers related to PPI in ES/SR projects. These may include personal, organisational or behavioural difficulties. 4. 'Best practice' is defined as the effective ways of meaningfully involving patients/public in the synthesis/review process, as recommended by the authors of included papers.
5. The terms 'evidence synthesis' and 'systematic review' are used interchangeably to explain the process of using a systematic, explicit method to review evidence from primary research; although there are debates surrounding the use of these terms. 21

| Search strategy
To define the key elements of the research questions and inform the standardisation of the search strategy, the SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) tool was chosen 22 (see Table 1) to identify relevant qualitative and/or mixed-method studies. However, due to poor indexing of qualitative studies in databases, the use of descriptive titles/abstracts, poor description of qualitative methods 23,24 and limited time, the decision was made that the search strategy would be constructed by identifying and combining keywords/concepts supporting this review's aims/objectives 25 the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre). Furthermore, the reference lists of key studies were screened for additional articles. 27 There were no restrictions on publication status, language or publication date before conducting the searches to avoid bias. 28 See Supporting Information: File S1 for search strategy details.

| Study selection
Two independent reviewers (E. A. M. and N. A.) systematically selected studies for inclusion via the Covidence software, 29 based on the eligibility criteria outlined below. Titles and abstracts of all studies were first reviewed, followed by a full-text review of eligible studies.
Any disagreements were resolved in consultation with a third reviewer (R. M.).
For studies to be included in this review, they had to meet the following criteria: 1. Studies or documents (e.g., reports from health institutions) which describe, reflect on, or evaluate the perspectives of patients/public and/or researchers on the benefits and challenges of PPI in individual ES/SR projects. Studies were excluded from this review if they described patients'/public's and/or researchers' involvement in other types of research (e.g., clinical trials), and/or were limited to research in social science. Commentaries, editorials, conference abstracts and letters were also excluded.

| Quality assessment
The Joanna Briggs Institute's (JBI) qualitative checklist was used to assess the methodological quality of included studies. 30 Quality of studies were graded as follows: 'High'-80% or greater; 'Moderate'greater than 50% but less than 80%; and 'Low'-less than 50%.
Quality assessment was conducted independently by two reviewers approach. 31

| Data extraction
A data extraction template was developed, piloted and appropriately refined using two included studies. Extracted data included author name (and year); study title; location/country; study aims/objectives; study design/type; health topic/focus; description of patients/public involved; stage, level, role, and/or method of PPI; description of the benefits and challenges of PPI from the perspectives of patients/ public and researchers; authors' recommendations for best practice or the challenges identified (see Table 3 and Supporting Information: Files S4 and S5).
To preserve the unique perspectives being presented by authors, original and quoted texts from the selected papers describing the benefits, challenges, and best practice(s) for PPI were directly extracted. Due to the possible distortion of data as 'findings' (e.g., when participants' data assumingly 'speak for themselves' without researchers' interpretation) and the different reporting styles in qualitative research, 32 'findings' from selected studies relating to the benefits, challenges or best practice were considered to be all original and quoted text were labelled under the 'Abstract' and/or 'Results' sections in the study reports. In instances where similar texts were provided by authors in both the 'Abstract' and 'Results' sections, the text from the 'Results' section was extracted-to enhance the contextual meaning of the data being extracted. Additionally, texts were included from the 'Discussion' section of study reports if they contained extra valuable information relating to benefits, challenges or best practices. Extraction was completed by E. A. M. and independently checked by N. A. Any disagreements were resolved in consultation with a third reviewer (R. M.).

| Data analysis and synthesis
We utilised the methods of thematic synthesis by Thomas and Table 2 summarises PPI in our review.

| Search results
Overall, 9923 titles and abstracts were screened, and 72 studies were independently assessed for eligibility during the full-text screening stage. A total of eight studies [35][36][37][38][39][40][41][42] were included in the final analysis (see Figure 1). Table 3 summarises the characteristics of the eight studies included in this review. Seven of these studies were entirely or mainly conducted within the United Kingdom. All included studies were qualitative, peer-reviewed articles that focussed on patientrelated or public health-related topics and were published between 2012 and 2021.

| Using the JBI qualitative checklist
Based on the calculation of the quality score for each included study, six studies were considered to be of high quality 35,[38][39][40][41][42] while the remaining two were of moderate quality 36,37 (see Figure 2).

| Using the GRADE-CERQual approach
Based on the assessment of methodological limitations, coherence, relevance and adequacy, the overall confidence in the findings of this review was rated as moderate. Thus, it is possible that the findings of this review reasonably represent the true phenomenon. The summary of qualitative findings using the T A B L E 2 Guidance for reporting involvement of PPI partner (GRIPP2 checklist).

Aim
Our aim was to synthesise qualitative evidence on the benefits, challenges and best practices for PPI in ES/SR projects from the perspectives of patients/public and researchers and work collaboratively with a PPI partner to analyse the findings, contribute to the writing of the report, and the publication and dissemination of findings.

Methods
The PPI partner was invited via a closed recruitment strategy. Authors (E. A. M. and R. M.) met virtually with M. S. to present the review and the journal reviewers' feedback and discuss whether there was agreed-upon interest to contribute to this review and how this involvement could meet everyone's needs. Several virtual consultative meetings and email communications between authors (E. A. M. and R. M.) and M. S. ensued. M. S. contributed to the final draft of the review at all stages of the writing of the report and the responses to the reviewers. Authors (E. A. M. and R. M.) were committed to supporting M. S.'s contributions. The PPI partner was included as a coauthor of this review due to her involvement and contributions which meet the ICMJE authorship criteria.

Study results
The PPI partner provided valuable input and understanding in interpreting the findings of the review and in providing context based on her experiences as a PPI coproducer of ES/SRs. The PPI partner led the development of the content for the GRIPP2 checklist.
Discussion and conclusions PPI partner highlighted important patients'/public's perspectives on involvement processes and provided valuable input in drawing appropriate conclusions from the evidence generated in this review.
Reflections/critical perspective The study authors, who did not have prior experience with PPI in ES/SRs, learned much from M. S. regarding the methods and value of PPI. As a result, they see the value of involving patients/public throughout the lifecycle of synthesis/review projects and are confident that this involvement can become an essential part of their research methods on ES/SRs. M. S. benefitted from collaborating with a team who are willing to try new approaches, which was of value to her goal of advancing coproduction of ES/SRs.
Abbreviations: ES, evidence synthesis; GRIPP2, Guidance for Reporting Involvement of Patients and the Public; PPI, patient and public involvement; SR, systematic reviews.
GRADE-CERQual approach is presented in Supporting Information: File S3.

Gaining knowledge
Patients/public who received training and support as part of the research process felt that they had either improved their level of knowledge about ES/SRs in general or about the specific topic/ study that they were involved in. 35

Enhancing relevance
Researchers felt that they were able to gain an accurate and deeper understanding of health issues related to the ES/SR project, 36,38,40 how patients/public think and feel, 38,40,42 and obtained a better understanding of the health needs of the patients/public. [36][37][38]40 Researchers agreed that involving patients/public enabled them to identify an unknown but critical perspective, which otherwise would have been missed by researchers, especially during the interpretation of synthesis/review findings. 38 Some patients/public also had difficulties in clarifying and/or undertaking their assigned roles/tasks during the review process 35,36,41 or did not have the opportunity to obtain the necessary individual feedback and support from researchers regarding their roles/tasks in the synthesis/review project. 35,36 Time Patients/public felt that the synthesis/review projects that they had been involved in had a longer duration than they had anticipated. 35,41 They felt that they did not have adequate time to prepare, and often had to overwork themselves to read papers and other research-related documents before meetings. 35,41 They highlighted the need for realistic deadlines and enough time during the synthesis/review process to guarantee their useful contributions/inputs to these projects. 35,41 Instructions came very late together with the 11 papers.
An immense amount of papers with too short a deadline.
I'm sorry but I had no time to read or prepare before the meeting. To be able to do this well together with our ordinary jobs for at least two months (11 papers!).

Thinking needs that time I think. (Patient research partner 2) 35
The length of time the research took was quite long…. 41 Additionally, the time-consuming nature of the synthesis/review projects made some patients/public lose personal interest in the current or future synthesis/review projects. 35,41 Low self-esteem Patients/public expressed self-doubt and queried the 'real' impact of their involvement on the synthesis/review project, either wholly 35,36,41 or in a specific way. 35,36 They expressed frustrations about their seeming lack of or inadequate knowledge regarding the health topic/intervention being assessed or the synthesis/review process, 35,36 and whether their views were fully considered by researchers in decision-making processes. 35,41 To be honest, I'm not quite sure. I don't know… if I was useful really, not in a general way but more specific. I

| The perspective of researchers
Time Researchers also felt that they did not have adequate time to develop PPI networks, 37,40 or to plan and conduct PPI in the ES/SR projects 36,39,41 Also, researchers reported that their interactions with patients/public were limited to the busy times of the project timetable, and this made it difficult for them to build good relationships and foster trust with patients/public. 36,37,40 We had no dedicated team member responsible for maintaining end-user relationships, and at the busiest times in the project timetable, there was little time to think about end-user involvement. This limited the opportunities for collaboration, most of the involvement being consultative in nature. It was also not possible for the researchers to develop a good rapport with end-users as the opportunities for relationship building were limited. 36 Furthermore, researchers expressed difficulties in involving patients/public at multiple stages in the synthesis/review process due to the duration of the synthesis/review projects. 37,40 Researchers felt that involving patients/public in the synthesis/review process was a complex task (especially during the stage of data synthesis) which demanded time. 39,41 Balancing inputs and managing relations and needs of patients/public and researchers relative to these projects. 36,37,40,41 Common to several of the end-user events, managing expectations and balancing the enthusiasm of end-users with a realisation of what was achievable within the project scope was difficult. As researchers, we did not always feel comfortable with this and were aware, at times, that our skills in this area may not be adequate. 36 Researchers felt that there were limited opportunities for patients/public to influence these projects through their inputs/ contributions. 36,41,42 In some instances, researchers experienced difficulties in managing discussions around sensitive topics which directly or indirectly affected patients/public. 36,40,42 Additionally, researchers had difficulties in sharing control, 37,40 and limited opportunities for building good relationships between patients/public and themselves to facilitate cooperation and coordination for mutual benefits. 36,41 Resources and training Researchers felt that they encountered difficulties in obtaining funding for ES/SR projects which involved patients/public, 37,40 although such projects were often seen by researchers to be resource-intensive. [36][37][38]40,41 They felt that they needed additional funding to ensure that PPI had a meaningful impact on the findings or synthesis/review project. 38,42 Finally, we hoped to involve young people at the end of our review to share our results but were unable to do this due to limited resources and time. 38 Moreover, researchers acknowledged that they sometimes lacked the requisite skills to properly involve patients/public in and/or effectively manage the synthesis/review process. 36,40 …As researchers, we did not always feel comfortable with this [managing expectations and balancing the enthusiasm of end-users] and were aware, at times, that our skills in this area may not be adequate. 36 They also agreed that the lack of or inadequate training and support for patients/public could limit their meaningful involvement in the synthesis/review process. 36 The authors also recommended that there is the need to recruit a dedicated PPI coordinator to liaise between researchers and patients/public. 37,40,42 Consider the involvement of a dedicated PPIE [patient and public involvement and engagement] facilitator. 42 Additionally, researchers must adopt a flexible approach to PPI and define, clarify and document their level of involvement throughout the synthesis/review project. 36,40 Some authors agreed that patients/public should be given the opportunity to withdraw from these projects at any time after researchers have explained the nature of their involvement to them. 40 The authors also suggested that patients/public should be provided with different communication avenues (including emails, post, telephone, online and written), with discussions employing either an individual or a focus group format. 35,37,40,42 Give members different ways of expressing their opinion and any concerns (written, online, withingroup, individually). 37 Utilisation of focus group with meaningful number of participants to discuss review findings. 35 3.6.4 | Need to foster positive working relationships Researchers must acknowledge the need to manage power imbalances between patients/public and themselves throughout the process, as well as resolve conflicts between patients/public involved. 36,37,40,42 Recognise power relations can be an issue to manage.
Clearly recognise and appreciate PPIE [patient and public involvement and engagement] members expertise. 37 Authors agreed that researchers ought to value the patients/ public involved and acknowledge their potential inputs/contributions to the synthesis/review process. 35,37,40 However, researchers must properly manage the expectations of patients/public and ensure that they are feasible for the project. 36

| Strengths
This review has a number of strengths. It utilised the methods of thematic synthesis by Thomas and Harden, 33 a widely used procedure in the health sciences which offers a transparent process in synthesising data from qualitative studies with varying reporting styles. Included studies had moderate-to-high methodological quality, and the use of the GRADE-CERQual approach suggests that there is good confidence in the review findings to support its use in decision-making processes, including policy formulation and guideline development.

| Limitations
The review also has some limitations. Firstly, the lack of a common reporting style (and other reporting limitations) posed difficulties in locating and extracting the 'right' data from the selected studies. As such, some relevant data may have been missed at the data extraction stage, although data extraction was undertaken by two independent reviewers to minimise this risk. Again, there were difficulties in distinguishing between the evidence from patients'/ public's perspectives due to the mixed reporting of these points of view in some of the included primary studies; hence, both groups were treated as 'patients and the public' 8 in this review. Additionally, studies published in languages other than English were excluded, and this might lead to language bias. Finally, most included studies were conducted in the United Kingdom, and this limits the generalisability/ transferability of findings to other settings, particularly to low-and middle-income countries (LMICs).

| Implications for research
Future studies must be targeted at the utilisation of existing frameworks (e.g., ACTIVE framework) by researchers to help describe the approaches and methods for involving patients/public in ES/SRs.
Additionally, future studies from diverse settings (including LMICs) should aim to publish the benefits, challenges, context and processes of involving patients/public to help locate essential components which facilitate meaningful PPI in synthesis/review projects (e.g., recruitment strategies, medium of communication, etc.).

| Implications for practice
The evidence from this review suggests the need for ES/SR-related organisations to utilise context-specific resources/materials on the involvement of patients/public in individual synthesis/review projects, including compensation guidelines and nonpecuniary methods for acknowledging patients'/public's skills and time, budgeting for involvement activities, and so forth. Additionally, guidance on context-specific training and support for both researchers and patients/public should include who receives what specific type of training, method of training, organising training events, ways to build a good rapport between researchers and patients/public, and so forth.
For organisations undertaking rapid reviews (RRs), the stage and level of involvement of patients/public may depend on the interests and abilities of the patients/public recruited, the project timeline, available funding, and whether patients/public will be involved in a single RR or multiple RRs in a related domain of healthcare.

| CONCLUSION
This qualitative SR provides a comprehensive overview of the benefits, challenges and best practices for PPI in ES/SR projects from the perspectives of both patients/public and researchers. PPI