SEARCH

SEARCH BY CITATION

Keywords:

  • health and social care research;
  • impact on research;
  • patient and public involvement;
  • user involvement

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Background

There is an increasing international interest in patient and public involvement (PPI) in research, yet relatively little robust evidence exists about its impact on health and social care research.

Objective

To identify the impact of patient and public involvement on health and social care research.

Design

A systematic search of electronic databases and health libraries was undertaken from 1995 to 2009. Data were extracted and quality assessed utilizing the guidelines of the NHS Centre for Reviews and Dissemination 2009 and the Critical Appraisal Skills Programme (CASP). Grey literature was assessed using the Dixon-Woods et al. (2005) checklist.

Inclusion criteria

All study types that reported the impact PPI had on the health and/or social care research study.

Main results

A total of 66 studies reporting the impact of PPI on health and social care research were included. The positive impacts identified enhanced the quality and appropriateness of research. Impacts were reported for all stages of research, including the development of user-focused research objectives, development of user-relevant research questions, development of user-friendly information, questionnaires and interview schedules, more appropriate recruitment strategies for studies, consumer-focused interpretation of data and enhanced implementation and dissemination of study results. Some challenging impacts were also identified.

Conclusion

This study provides the first international evidence of PPI impact that has emerged at all key stages of the research process. However, much of the evidence base concerning impact remains weak and needs significant enhancement in the next decade.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Patient and public involvement is thought to improve the way the research is prioritized, commissioned, undertaken, communicated and used.[1] Active involvement of service users in research can lead to research of greater quality and relevance owing to the unique perspective that users can bring to a research project.[2-5] Interest in involvement has expanded internationally with many countries now actively involving users in research. In recent years, researchers have been encouraged to develop systems and processes to involve patients and the public in research[6, 7] supported by a developing infrastructure provided by INVOLVE, an organization in the UK that promotes public involvement in research, and the Research Design Services in the UK. However, involvement in health research has rarely been systematically evaluated. Early work has concentrated on drawing out the extent of the policy and activity, rather than analysing its impact.[8-12] The interest that user involvement has attracted and the progress it has made now makes it timely to examine systematically the impact it makes on research and to consider the international implications for a future patient and public involvement (PPI) evidence base in health and social care.

Other reviews have identified the impacts of PPI in specific areas of health and social care research[13-17], and a structured review by INVOLVE identified the themes over broader areas.[18] However, a systematic synthesis of the impact of PPI in health and social care research is needed. The overall objective of this systematic review is to examine the conceptualization, definition, measurement, impact, and outcomes of PPI in health and social care research. This paper reports the results in relation to the impact of PPI on health and social care research specifically. Impacts on individuals, organizations and communities involved in research are reported separately.[19]

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Study design

A systematic review method was adopted for the study, utilizing the principles and methods provided by the NHS Centre for Reviews and Dissemination guidelines.[20]

Data sources

Systematic searches were undertaken from 1995 to April 2009 in the following databases: medical literature (Medline, Embase, PsychINFO, Cochrane library), social care literature (Assia), nursing literature (CINAHL) and healthcare management information consortium (HMIC and HELMIS). A comprehensive search combined sets of terms including and relating to patient and public involvement (consumer, citizen, client, carer, lay, service users, survivor, stakeholder, family, relative); health and social care (public health, primary health); type of involvement (particp*, collaborat*, engage*, partner*, consult*, user led, consumer panel, advisory board, evaluat*) and PPI outcomes (impact, effect, adapt*, change, develop*, focus group). Hand searching of reference lists of papers and hand searching of the journal ‘Health Expectations’ was conducted. Grey literature was searched using the libraries InvoNet and NHS Evidence. Grey literature was also obtained by contact with key experts in the field. INVOLVE were contacted and their library searched.

Study selection

All study types that were in English language and reported data on the involvement of adult service users were included. The first 10% of abstracts were reviewed by two reviewers (JB, SS). As agreement between reviewers on included papers was high (94%), and because of the large number of papers involved in this process, the rest of the abstracts were reviewed by one reviewer and checked by a second reviewer.

Quality assessment

The methodological quality of published studies was assessed using the Critical Assessment Skills Programme (CASP) checklist.[21] Grey literature was assessed using the Dixon-Woods et al. checklist[22] as used by Hubbard et al.[14] to review grey literature on involving people affected by cancer.

Data extraction and synthesis

Data were extracted into tables and categorized according to the reported impact of PPI on research and research processes. For each paper, the key data extracted included the following: the author; the year and country; the aims of the study; the methods used for the study; methods used for the patient and public involvement in the study; whether the PPI was a consultation, collaboration or user-led; the number of service users involved; the impacts of the PPI on the study; the limitations of the study and the quality assessment of the study. A qualitative narrative synthesis of the data was performed, which involved familiarization with the papers and the identification of emergent themes.[23]

The narrative synthesis followed the CRD for systematic reviews guidelines by first developing a theory of how the PPI impacts on the study. This was achieved by developing a preliminary synthesis of the included studies, exploring relationships between the studies and finally assessing the robustness of the synthesis by conducting an expert seminar with service users and researchers involved in PPI to provide a critical reflection of the results.[20]

User involvement in study

Three users were recruited to the advisory board of this study and they commented on the design, methodology and analysis in the systematic review. The service users also contributed by commenting on the focus of the study and had an important impact on the analysis and interpretation of data, primarily through an expert seminar that was held at the end of the project. This seminar included 24 service users and individuals who work in the area of PPI and provided an opportunity for users, researchers and others involved in PPI to discuss the emerging findings from the systematic review and to add their interpretations and perspectives.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

The total number of titles in the first electronic search were 13 890. After the first title and abstract review, this was reduced to 253 papers. After reading the full article, a total of 55 papers met the inclusion criteria. A further three papers were included from hand-searching the journal Health Expectations, and eight reports were obtained from the grey literature searches. A total of 66 papers and reports that described the impacts of PPI on health and social care research were included.

The majority of published papers were quality-assessed as adequate. Only 13 papers were quality-assessed as partially adequate and these included five case studies or case series studies,[24-28] five qualitative studies[29-33] and one review of the literature[13] Five non-published reports were included and these were all quality-assessed as five[17] or four[34-37] using the Dixon-Woods quality assessment checklist for grey literature. Those papers that were quality-assessed as not adequate on the CASP checklist or three or less on the Dixon-Woods checklist were excluded. Data extraction tables and quality assessment are reported in the full report for this study.[19] Figure 1 below shows the search results.

image

Figure 1. Search results.

Download figure to PowerPoint

Of the 66 studies, two were randomized-controlled trials (RCTs), one was a pre-test/post-test study, one was a cohort, 46 were qualitative studies, nine were cross-sectional, five were case studies and two were case series. The majority of studies were from the UK (n = 45), but evidence was also reported from the USA (n = 12), Canada (n = 5), the Netherlands (n = 2) and Australia (n = 2). The impacts are reported by stages of the research process, from the initial stages of research development, conducting the research project through to implementation and dissemination of the findings. Both beneficial and challenging impacts are reported.

Beneficial impacts of PPI on research and research process

Initial stages of research

During the initial stages of developing and setting up a research programme, there was evidence of positive impacts of user involvement, with users helping to identify user-relevant topics for the research agenda,[2, 38, 39, 29, 40-46] prioritizing topics for the research agenda[17, 20, 42, 44, 47, 48] and developing the patient-relevant commissioning briefs.[43, 44, 34, 49]. Studies reported the involvement of users in the development of research topics that were grounded in day-to-day reality of users' experiences. Examples of this included the following: involving mothers of pre-school or primary school aged children to develop topics of research to improve health and well-being of families and children before school age; involving stroke patients to identify a research study about awareness and knowledge of stroke and stroke risk and involving mental health users in research on adult mental health services.[49-52] Other examples were reported in a range of research areas including diabetes, rheumatology, spinal cord injury, blind and partially sighted, older people, health technology, biomedical research and Cochrane reviews.[29, 40, 42, 53, 24, 25, 54, 55] The involvement of users helped develop user-relevant research questions. There was also evidence of users being recruited onto steering groups or advisory groups to help advise on research studies, for example, to direct primary health and social care research agenda in one city in the UK, by sitting on a steering group for a randomized control trial of HRT and breast cancer and by sitting on the steering group for research into Paget's disease.[17, 29, 44, 48, 35, 56, 57, 30, 58-60] Panels of consumers also helped funders to identify which research proposals should be accepted.[61]

Undertaking research

During the development of the research protocols, users offered pragmatic criticism and commented on the extent to which they perceived the research to be relevant or appropriate to users.[6, 55, 62-65] Examples included identifying cultural issues that should be taken into account when designing the study,[17, 55, 62] identifying patient important outcome measures, solving issues around how to get informed consent[63] and advising on the appropriateness of design from the user perspective.[55]

Input from users also helped adapt academic language to suit lay audience,[15, 39, 36] by improving the wording of patient information and invitation letters,[66] and improving the sensitivity of the wording of the information to ensure cultural appropriateness.[15, 55]

There was also evidence that users assisted in recruitment by providing greater access to the research community[42, 45, 36, 67-69] and by identifying the effective ways of accessing participants.[8, 17, 51, 57, 64, 31, 70, 71, 32, 72] Many of the user associations, such as the Spinal Cord Injury Association (SCIA) and the National Association for the Relief of Paget's disease (NARPD) have helped in the identification of a study sample.[42, 58] Service users can also help participants to become more informed about the investigation and treatment of disease, which may lead to better informed consent.[55, 58, 73, 67, 69, 70]

Furthermore, studies reported that user involvement helped in assessing the appropriateness of research instruments from a community-perspective, in order to develop user-relevant tools,[8, 39, 29, 50, 51, 64, 73, 32, 26, 74] and assisted in the development of questionnaire/interview schedules by identifying lines of inquiry not previously considered, helping with the wording of questions, assisting with the timing of interventions and ensuring questions asked were acceptable to the local community.[29, 49, 57, 64, 66, 68] In one study, users helped researchers gain invaluable cultural perspectives of diabetes, particularly how diabetes was often concealed in certain communities because of social stigma, which helped develop a more appropriate study protocol.[55] User involvement in the interview process also had an impact. While interviewing or collecting data in face to face interviews, studies show that deeper and more personal insights were gained, because of the rapport and empathy users developed with participants, putting participants at ease and providing a greater understanding of the encounter.[75, 36, 67, 72, 76, 33] One study found that users knew the right questions to ask of participants, as the issues had a personal resonance for them.[42] Three studies that recruited interviewers from the mental health community and one from the IDUs community reported there was a more honest flow of information during these interviews.[67, 76, 33, 77]

Analysis and write-up stage of study

During the data analysis, the review showed that the involvement of users helped to ensure that emerging themes and trends were interpreted from the different lay perspectives as well as from the academic and clinical perspectives, and assisted in identifying research gaps.[75, 50, 52, 57, 64, 36, 73, 32, 26, 27] Final research reports benefited from being grounded in user experiences, by providing a wider, more relevant viewpoint,[29] by providing cultural relevance[31] and by giving the results better credibility with stakeholders.[69]

Dissemination and implementation

The evidence reports that involvement of users may achieve better dissemination and implementation of research findings because of the dedication and influence of users in the community.[17, 39, 45, 75, 58, 64, 32, 78] Studies showed how users created a cohort of advocates for implementation and dissemination of results.[8, 50, 51, 57, 58] Users delivered more poignant messages at conferences and through newsletters by relating the findings to their own experiences[15, 58] and presenting them in a more lay user-friendly way.[49]

Challenging impacts of PPI on research

Initial stages of research

During the initial stages of setting up a research programme, studies reported more challenging impacts of PPI on both researchers and service users. Incorporating user views into the research agenda may lead to divergence from scientific methods and cause ethical dilemmas during the protocol design. Compromises may have to be reached to ensure user views are incorporated in a logical manner.[63] Two studies reported where community concerns conflicted with the research methods, leading to a no placebo arm to ensure that patients knew if they were in the treatment arm or not.[59, 63] While this may have impacted on the robustness of the study design for researchers, it provided a more ethical study design for users. This can provide an important challenge for researchers and highlight potential tensions between academic criteria of good quality research compared with the user perspective of what constitutes appropriate research.[25] It is the researcher's role to ensure any study is of high quality scientifically and to explain and justify the research design and negotiate changes that make the study more acceptable to service users without compromising its robustness or validity. Offering service user training in research methodology may help maximize the service user involvement and empower service users in their contributions to the design of the study, providing service users with the tools to discuss outcomes and formulate questions rather than limiting their involvement to accounts of their experiences[4, 10, 39, 79]

There is also evidence of researchers' tokenistic attitude towards PPI, for example, researchers involving users to comply with policy[15, 57, 28] or because they do not really understand the contribution PPI could bring to the research.[10] This type of involvement can result in users' input being devalued by the research team and a poor experience for users. Furthermore, researchers often struggle with relinquishing control over the study that has led to conflict between parties.[48, 25, 67, 32, 37] For example, in a research study that assessed the needs of elderly people, difficulties emerged in the partnership between researchers and users and led to the ‘turning upside down of existing power relationships’. Academic researchers and health professionals have traditionally had control over what is researched, and user involvement can change this balance. While this can be a difficult adaptation for researchers, it has the potential to offer valuable contributions to the research.

Data collection stage of research

Studies reported the challenges of recruiting service users from a diverse range of users. For example, studies reported difficulties recruiting hard-to-reach groups, such as minoritiy ethnic groups, older people, people with disabilities, users who felt they have nothing to contribute and users who suffer anxiety concerning group situations.[42, 65, 69, 37] Even after service users had agreed to be involved in the study, low attendance rates in research meetings caused further problems.[60, 69, 80] Service users need to feel their involvement is being valued and are made to feel confident in making changes. In one study, an informed consent document was developed by researchers and then presented to service users to adjust. There was no significant difference in participant's understanding of the study reported between the two consent documents, which may have been because users did not feel they could make a substantial change to a pre-existing informed consent form.[81] The relevance and understanding of the materials to the public may have been better if the users were involved in the initial development of the consent form or if the users had felt sufficiently empowered to raise their concerns earlier.

Uncertainty about how confidential information provided in meetings would be treated caused anxieties for some users.[29, 42, 73, 37] During meetings, issues of patient confidentially were sometimes difficult to maintain, for example, in a study exploring the views of people affected by cancer, users discussed their own treatment and care during steering group meetings, which raised a range of ethical issues.[73]

Users sometimes challenged the methods used by researchers causing conflict within the study. During the interviewing phase of one study, users felt restricted by the interview schedule and departed from it when they felt it was appropriate, leading to rich, in-depth data that may not have been collected from using the interview schedule. However, this departure from the interview schedule raised issues of academic integrity for researchers.[25] Again, this could be an issue the researcher could have avoided if they had explained and justified the research design more clearly and conferred with the users to arrive at a design acceptable to both researchers and service users.

When identifying research topics, there could be dominant parties within the group that leads to a potential for over-emphasizing of problems that are of particular concern to the dominant force.[43, 72] These meetings should be carefully managed to allow a fair representation of voices. Furthermore, care should be taken in providing clear instructions for the meeting to avoid sessions being over-rided by personal experience stories.[79] In addition, some researchers have raised concerns about users losing their objectivity and becoming ‘professionalized’ as the boundaries between lay researchers and academic researchers become more blurred over the lifetime of the project.[80, 82] Such complexity highlights the need to consider the context and process of involvement when evaluating impact.

Dissemination and implementation

While most studies report on the beneficial impacts of PPI during dissemination and implementation, there were some initial insights into the perceived challenges of publishing in academic journals. One study investigated whether researchers publishing in international general medical journals had actively involved consumers in their research and dissemination. However, involvement was reported as being integral to the research undertaken in just six of 200 originally published papers. The researchers reported the following challenges that prevented them from involving users: limited word counts prevented documentation of PPI in journal articles; results were not perceived as important; and concern that the users involved may disseminate the results before they have been written up and published in academic journals.[82]

Time and cost

Practical aspects of planning, collaborating with users and managing user involvement in the research can be timely and costly.[4, 39, 42, 36, 73, 67, 69, 72, 27] The evidence reports the importance of developing good working relationships with communities and good links to service user organizations and reports the importance of education and training of users.[83] However, this may be difficult within the time and funding limitations of a study.[75, 83] One study reported that the short timescale given to researchers to set up a user group led to a lack of diversity within the group.[56] Further, time delays occurred in another study owing to the conflicting time frames of researchers and users[42] and failure to allow users a realistic amount of time to read documentation prior to meetings.[84]

Running and maintaining the user membership, existing work commitments and the need to account for the health status of those involved increased the workload of the researchers.[42, 45, 83] Projects need to incorporate these additional timescales in research proposals and funders need to be willing to fund this activity.

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

This systematic review provides a systematic synthesis of the evidence of the PPI impact on health and social care research summarized in Table 1. While other reviews have identified themes and categories of impact,[18, 85] this is the first international systematic review reporting the impact of PPI across all of health and social care research. This review reports clear evidence that patient and public involvement can have positive impact on research, enhancing the quality of research and ensuring its appropriateness and relevance. However, challenging impacts are also reported and, although the evidence base for these is much smaller, a strategy to minimize these challenges should be considered early on in the research process. A number of trends were identified by the review. Patient and public involvement may have a more positive impact when service users are involved throughout the study and when involved as partners in the research team, although this needs to be explored further in future studies. Challenges appeared to be reported in studies where service users were involved sporadically in the study with no clear role.

Table 1. Summary of evidence by themes from systematic review
  1. PPI, patient and public involvement.

Beneficial impacts
Initial stages of research: PPI helped identify relevant topics for the research agenda, assisted in prioritizing topics for the research agenda and provided pragmatic criticism of the research protocol in perceiving whether research was relevant or appropriate to users
Undertaking research: PPI helped assess the appropriateness, wording and timing of research instruments (e.g. questionnaires, interview schedules) to the community and helped adapt the language of the instruments and information to suit the lay audience. PPI also assisted with recruitment to the study and improved response rates. Furthermore, PPI helped gain deeper and more personal insights because of the rapport users had with participants
Analysis and write-up: PPI helped ensure emerging themes and trends were interpreted from the user perspective as well as the academic researcher perspective, assisted in identifying relevant knowledge gaps, and final research reports benefited from being grounded in user experiences
Dissemination and implementation: PPI helped with the dissemination and implementation of research findings owing to the dedication to and influence of users to the community. Studies reported that dissemination was delivered in a more poignant and user-friendly way
Challenging impacts
Initial stages of research: Studies indicated that PPI led to scientific and ethical conflict in protocol design, which may have been due to a lack of knowledge and understanding of PPI. PPI may lead to a tokenistic nature of users' involvement and can cause power struggles between researchers and users
Data collection stage of research: PPI studies have reported the difficulty in recruiting a diverse range and representative sample of users to a project, the difficulty in balancing traditional academic criteria for reliability and user perspectives in a protocol for research and the difficulty in maintaining user confidentiality within meetings, where users may discuss personal experiences
The challenges reported by researchers in running PPI focus groups included users influencing each other, which may result in an over-emphasizing of particular problems; groups being dominated by strong characters and their perspectives; groups being overshadowed by personal experience stories, when the aim was to identify research topics, and groups seen as a forum to get other people to accept their understanding of the disease
Dissemination and implementation: PPI led to research findings being disseminated before the academic papers are published, thereby jeopardizing academic publication
Time and Cost: PPI led to increased time and cost owing to the practical aspects of planning and managing the user involvement in the research, the time and cost of building up relationships within the community and setting up user groups, the training and education for both users and researchers and the additional time needed for users to read and comment on documentation

The impact of PPI in the initial stages of research, particularly around setting research agendas and research questions with users, helped to identify user-relevant topics for research agenda grounded in their own experiences. This represents a critical area for user involvement as it can shape an entire study and users may have more freedom to influence the aims and methods at this initial stage.[12] Where service users are involved later in a study, their influence on the focus of the study appears to be diminished, as might be expected.

The potential for PPI to assist with assessing the appropriateness of research tools is an important area. For example, PPI can lead to better wording in a questionnaire, identification of appropriate content, thus aiding content and face validity of measurement tools and aiding the identification of lines of enquiry not previously considered. In these ways, PPI is providing important contributions to improving the quality of the research process. As part of the appropriateness of research and research questions, PPI was also found to have an impact in ensuring the cultural relevance of studies and by providing a broader cultural understanding, which could inform protocol development.

There are also studies that support users as active researchers. Some studies found that deeper and more insightful data were gained within research interviews, possibly because of a better rapport between interviewer and interviewee. Overall, such findings reinforce the argument that user involvement helps to improve the quality of data and the relevance of research.

The analysis of study findings can be an important stage at which to involve service users as PPI can help to broaden interpretation of data, providing a different insight and helping to identify the aspects of research that have most relevance to users. The results of studies developed with users can also help with establishing the credibility of findings with stakeholders, particularly important when attempting to implement study findings. In addition, users can help to identify gaps in research that future studies need to address, can ensure that users have continued input into the research agenda, and can contribute to capacity development for continued user involvement. Committed service users can become advocates of the research findings, delivering a wide dissemination of results.

While many papers reported on the beneficial impacts on research and the research process, fewer reported on the more challenging impacts. This may indicate that the benefits of PPI far outweigh the challenges of PPI, or it may indicate publication bias. Some of the challenges presented could be viewed from different perspectives, as one person's positive impact might be seen as someone else's challenge. We have tried to reflect the way in which they have been presented in the literature, but acknowledge the potential for different interpretations.

Many of the challenges occurred because of the problems of colliding worlds, where priorities, motivations and ways of working differ and science gets contested, causing conflict and power struggles between researchers and service users.[86] It is therefore vital that each member of the research team are clear of their specific roles and that each member of the team understands the distinct expertise that individuals bring to the team. For example, it is the researcher's role to ensure any study is of high quality scientifically. The service users' role is ‘equal but different’ to researchers in that their unique perspective of the lived experience of the condition under investigation is what brings added value. Service user involvement needs to be well planned, motivations discussed, individuals' roles defined, ways of working considered and guidance provided to both service users and researchers for sufficient understanding of the contribution that patient and public involvement can make to research.

Furthermore, there may be practical issues such as the difficulty recruiting a set of service users to be involved in the research, the long-term commitment needed from service users and the time and cost limits imposed on studies. The latter can often form an important barrier to activity and so potentially impede impact. Studies need to build in appropriate time, and funders and commissioners should acknowledge this need as part of providing an appropriate context for PPI, to create the conditions where involvement has the potential to have a positive impact.

Context and process of PPI

In synthesizing the data for this review, a user involvement workshop was held, where users contributed to the analysis of key themes. One of the important outcomes of this day and the continued involvement of the user group was the identification of the importance of context and process in the interpretation of impact.

The context refers to the environment in which PPI is undertaken, that is, whether the right conditions are in place for user involvement to work. It could include funding, policy, physical environment or the attitude of those involved. The process of involvement refers to more specific factors. For example, it could refer to the level of involvement that users have, how they are involved, when they are involved and what procedures are put in place to improve the likelihood of success (see Fig. 2). While the impact of PPI needs to be considered within the context and process, the current evidence base lacks such detail, making it difficult to synthesize the evidence in a more detailed and meaningful way. In some respects, this highlights the need for future reporting of impact to include detailed description of how, where and when the user involvement was conducted and to consider PPI as a complex intervention that requires appropriate evaluation.

image

Figure 2. Affects of Context and Process on impact of patient and public involvement in research.

Download figure to PowerPoint

The evidence demonstrates that the better the training, planning and procedures that are put in place, the clearer the definition of roles, the more positive the attitude towards PPI and the greater the trust and respect that parties (users, researchers, clinicians, funders, policy makers) have with each other which may lead to more potential for beneficial impact. The less involved the users are in the research, for example, if there is a lack of training, poor planning and unclear procedures and roles put in place, a more negative attitude, and a lack of trust and respect between parties, the more challenging the involvement can be and possibly less chance of beneficial impact.

The nature of impact

In reviewing the impact of PPI on research, it is important to consider the limitations of the evidence base. While the systematic review methodology identified important data of impacts on research, these data were often brief, lacked detail and could be reported anywhere in the paper, not necessarily in the results section. The content validity of the reporting was also unclear, so the reader is unaware whether all aspects of impact are reported, or only certain ones. It is very likely that many impacts have gone unreported and greater consistency in reporting the full range of impacts identified in studies is required. Furthermore, most studies utilized narrative descriptions of data and none had attempted any quantitative measurement, reflecting the lack of robust tools specifically developed to provide a measure of the extent of impact. For this reason, it was difficult to quality-assess the papers, and the review took an inclusive approach by including all papers that stated clear aims and methodology and were set out as a research paper. The paucity of some experimental studies made it difficult to weight the data, as is customary in systematic reviews. As a result of this study, guidance is currently being developed in collaboration with EQUATOR to provide CONSORT-style guidance on how to report the impact of PPI.[87]

This review also has some limitations that relate to the nature of the PPI evidence base. A lack of MESh terms for PPI in the electronic databases used limited the searches. Furthermore, the variability of key words in the papers resulted in long and complex search strings. Once the papers were selected, it was difficult to measure the quality of the PPI activity because the assessment tools measured the quality of the main study rather than the quality of PPI activity.

In conclusion, this review has been helpful in identifying the current range of impacts of PPI on health and social care research, highlighting the positive impacts that PPI can have on a study. Challenges of PPI may be avoided through clear planning of the PPI activity in the early-planning stages of the proposed study. Overall, the evidence base is still relatively weak and requires further substantive development in terms of the way in which impact is reported, a clearer conceptual understanding of the nature of impacts and methods for assessing impacts both qualitatively and quantitatively. This can be achieved through consort-like guidance to improve the quality of reporting and to strengthen the PPI evidence base.[87] We will then be in a better position to evaluate what works, for whom and in what circumstances.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

We would like to acknowledge the advisory board that assisted through the project. They were Philippa Yeeles from the UKCRC, Rosemary Barber from Sheffield University, Mary-Rose Tarpey from INVOLVE, Jacqueline Chandler Oatts from the Cochrane Collaboration and Mark Petticrew from the London School of Hygiene and Tropical Medicine. We would also like to thank the UK CRC for funding this systematic review, Diane Clay, an Information specialist with Warwick University Medical School for her help with the literature searches and Professor Kate Seers for supporting the project through the RCN RI at the University of Warwick.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  • 1
    Involve. Colliding Worlds – report of the experiences service user involvement in research, 2004. Available at: www.invo.org.uk, accessed 10 June 2009.
  • 2
    Entwistle VA, Renfrew MJ, Yearley S, Forrester J, Lamont T. Lay perspectives: advantages for health research. British Medical Journal, 1998; 316: 463466.
  • 3
    Chalmers I. What do I want from health research and researchers when I am a patient?. British Medical Journal, 1995; 310: 13151318.
  • 4
    Oliver S. How can health service users contribute to the NHS research and development agenda?. British Medical Journal, 1995; 310: 13181320.
  • 5
    Goodare H, Smith R. The rights of patients in research. British Medical Journal, 1995; 310: 12771278.
  • 6
    Staniszewska S, Jones N, Marshall S, Newburn M. User involvement in the development of a research bid: barriers, enablers and impacts. Health Expectations, 2007; 10: 173183. (1369–6513).
  • 7
    Department of Health. Local Government and Public Involvement Act 2007. Available at: http://www.dh.gov.uk/publications, accessed 1 June 2009.
  • 8
    Hanley B, Truesdale A, King A, Elbourne D, Chalmers I. Involving consumers in designing, conducting, and interpreting randomised controlled trials: questionnaire survey. British Medical Journal, 2001; 322: 519523.
  • 9
    Dixon P, Peart E, Carr-Hill R. A Database of Examples of Consumer Involvement in Research. York: The University of York Centre for Health Economics, 1999.
  • 10
    Telford R, Beverley CA, Cooper CL, Boote JD. Consumer involvement in health research: fact or fiction? British Journal of Clinical Governance, 2002; 7: 92103.
  • 11
    Blaxter M. Consumer Issues Within the NHS: An R & D Contribution to Consumer Involvement in the NHS. London: Department of Health, 1995.
  • 12
    Oliver S, Buchanan P. Examples of lay involvement in research and development. EPI-centre, Social Science Research Unit, London University Institute of Education, 1997.
  • 13
    Abelson J, Giacomini M, Lehoux P, Gauvin F. Bringing ‘the public’ into health technology assessment and coverage policy decisions: from principles to practice. Health Policy, 2007; 82: 3750.
  • 14
    Hubbard G, Kidd L, Donaghy E, McDonald C, Kearney N. A review of literature about involving people affected by cancer in research, policy and planning and practice. Patient Education and Counseling, 2007; 65: 2133.
  • 15
    Smith E, Ross F, Donovan S et al. Service user involvement in nursing, midwifery and health visiting research: a review of evidence and practice. International Journal of Nursing Studies, 2008; 45: 298315.
  • 16
    Gilbert T. Involving people with learning disabilities in research: issues and possibilities. Health & Social Care in the Community, 2004; 12: 298308.
  • 17
    Viswanathan M, Ammerman A, Eng E et al. Community-based participatory research: Assessing the evidence. Evidence Report/Technology Assessment (Summary), 2004; 99: 18.
  • 18
    Staley K. Exploring Impact: Public involvement in NHS, Public Health and Social Care Research. Available at: http://www.invo.org.uk/Resources.asp, accessed 1 October 2009.
  • 19
    Brett J, Staniszewska S, Mockford C, Seers K, Herron Marx S, Bayliss H. Systematic Review of the Conceptualization, Measurement, Impact and Outcomes of Patient and Public Involvement in Health and Social Care Research. London: UKCRC, 2010.
  • 20
    Centre for Dissemination and Reviews (CRD). Systematic Reviews: CRD's Guidance for Undertaking Reviews in Health Care. York: Centre for Dissemination and Reviews (CRD), 2009.
  • 21
    Critical Appraisal Skill Programme. Available at: www.phru.nhs.uk/Pages/PHD/CASP.htm, accessed 1 April 2009.
  • 22
    Dixon-Woods M, Kirk D, Agarwal S et al. Vulnerable Groups and Access, to Health Care: A Critical Interpretative Review. London: National Co-ordinating Centre for NHS Service, 2005.
  • 23
    Popay J, Roberts H, Sowden A et al. Developing guidance on conduct of narrative synthesis in systematic reviews. Journal of Epidemiology and Community Health, 2005; 59 (suppl 11): A7.
  • 24
    Royle J, Oliver S. Consumer involvement in the health technology assessment program. International Journal of Technology Assessment in Health Care, 2004; 20: 493497.
  • 25
    Reed J, Weiner R, Cook G. Partnership research with older people – moving towards making the rhetoric a reality. Journal of Clinical Nursing, 2004; 13: 310.
  • 26
    Cashman SB, Adeky S, Allen AJ III et al. The power and the promise: working with communities to analyze data, interpret findings, and get to outcomes. American Journal of Public Health, 2008; 98: 14071417.
  • 27
    Trivedi P, Wykes T. From passive subjects to equal partners: qualitative review of user involvement in research. British Journal of Psychiatry, 2002; 181: 468472.
  • 28
    Minogue V, Boness J, Brown A, Girdlestone J. The impact of service user involvement in research. International Journal of Health Care Quality Assurance, 2005; 18: 103112.
  • 29
    Hewlett S, de Wit M, Richards P et al. Patients and professionals as research partners: challenges, practicalities and benefits. Arthritis & Rheumatism, 2006; 55: 676680.
  • 30
    Menon D, Stafinski T. Engaging the public in priority-setting for health technology assessment: findings from a citizens' jury. Health Expectations, 2008; 11: 282293.
  • 31
    Savage C, Xu Y, Lee R, Rose B, Kappesser M, Anthony J. A case study in the use of community-based participatory research in public health nursing. Public Health Nursing, 2006; 23: 472478.
  • 32
    Minkler M, Fadem P, Perry M, Blum K, Moore L, Rogers J. Ethical dilemmas in participatory action research: a case study from the disability community. Health Education & Behavior, 2002; 29: 1429.
  • 33
    Godfrey M. More than ‘involvement’: how commissioning user interviewers in the research process begins to change the balance of power. Practice, 2004; 16: 223231.
  • 34
    Oliver S, Armes D, Gyte G. Evaluation of public influence on the NHS Health Technology Assessment Programme. Social Science Research Unit, University of London, 2006.
  • 35
    UKCRC & TwoCan Associates. An Evaluation of the Process and Impact of Patient and Public Involvement in the Advisory Groups of the UK Clinical Research Collaboration. London: UK Clinical Research Collaboration, 2009.
  • 36
    Faulkner A. Beyond our Expectations: A Report of the Experiences of Involving Service Users in Forensic Mental Health Research. London: National Programme on Forensic Mental Health R&D, Department of Health, 2006.
  • 37
    Sainsbury Centre for Mental Health. A Review of Service User Involvement in Prison Mental Health Research. London: Sainsbury Centre for Mental Health, 2008.
  • 38
    Lindenmeyer A, Hearnshaw H, Sturt J, Ormerod R, Aitchison G. Assessment of the benefits of user involvement in health research from the Warwick Diabetes Care Research User Group: a qualitative case study. Health Expectations, 2007; 10: 268277.
  • 39
    Shah SG, Robinson I. Benefits of and barriers to involving users in medical device technology development and evaluation. International Journal of Technology Assessment in Health Care, 2007; 23: 131137.
  • 40
    Howe A, MacDonald H, Barrett B, Little B. Ensuring public and patient participation in research: a case study in infrastructure development in one UK Research and Development consortium. Primary Health Care Research and Development, 2006; 7: 6067.
  • 41
    Nilsen E, Myrhaug H, Johansen M, Oliver S, Oxman A. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database of Systematic Reviews, 2006; 3: 131.
  • 42
    Abma T. Patient participation in health research: research with and for people with spinal cord injuries. Qualitative Health Research, 2005; 15: 13101328.
  • 43
    Caron-Flinterman JF, Broerse JEW, Teerling J, Bunders JFG. Patients' priorities concerning health research: the case of asthma and COPD research in the Netherlands. Health Expectations, 2005; 8: 253263.
  • 44
    O'Donnell M, Entwistle V. Consumer involvement in decisions about what health-related research is funded. Health Policy, 2004; 70: 281290.
  • 45
    Rhodes P, Nocon A, Booth M et al. A service users' research advisory group from the perspectives of both service users and researchers. Health & Social Care in the Community, 2002; 10: 402409.
  • 46
    Kelson M. National Sentinel Audits Involving Older People: A Guide to Involving Older People in Local Audit Activity. London: College of Health, 1999.
  • 47
    Hailey D, Nordwall M. Survey on the involvement of consumers in health technology assessment programs. International Journal of Technology Assessment in Health Care, 2006; 22: 497499.
  • 48
    McCormick S, Brody J, Brown P, Polk R. Public involvement in breast cancer research: ananalysis and model for future research.International Journal of Health Services: Planning, Administration, Evaluation, 2004; 34: 625646.
  • 49
    Morgan L, Chambers R, Banerji J, Gater J, Jordan J. Consumers leading public consultation: the general public‟s knowledge of stroke. Family Practice, 2005; 22: 814.
  • 50
    Rowe A. The effect of involvement in participatory research on parent researchers in a Sure Start programme. Health & Social Care in the Community, 2006; 14: 465473.
  • 51
    Barnard A, Carter M, Britten N, Purtell R, Wyatt K, Ellis A. The PC11 Report. An evaluation of consumer involvement in the London Primary Care Studies Programme. Peninsula Medical School, Exeter, UK, 2005.
  • 52
    Clark M, Glasby J, Lester H. Cases for change: user involvement in mental health services and research. Research Policy and Planning, 2004; 22: 3138.
  • 53
    Caron-Flinterman JF, Broerse JEW, Teerling J et al. Stakeholder participation in health research agenda setting: the case of asthma and COPD research in the Netherlands. Science and Public Policy, 2006; 33: 291304.
  • 54
    Kelson M. User involvement in clinicalaudit: a review of developments and issues of good practice. Journal of Evaluation in Clinical Practice, 1996; 2: 97109.
  • 55
    Burrus BB, Liburd LC, Burroughs A. Maximizing participation by black Americans in population-based diabetes research: the Project DIRECT pilot experience. Journal of Community Health, 1998; 23: 1527.
  • 56
    Gooberman-Hill R, Horwood J, Calnan M. Citizens' juries in planning research priorities: process, engagement and outcome. Health Expectations, 2008; 11: 272281.
  • 57
    Wyatt K, Carter M, Mahtani V, Barnard A, Hawton A, Britten N. The impact of consumer involvement in research: an evaluation of consumer involvement in the London Primary Care Studies Programme. Family Practice, 2008; 25: 154161.
  • 58
    Langston AL, McCallum M, Campbell MK, Robertson C, Ralston SH. An integrated approach to consumer representation and involvement in a multicentre randomized controlled trial. Clinical Trials, 2005; 2: 8087.
  • 59
    Marsden J, Bradburn J. Patient and clinician collaboration in the design of a national randomized breast cancer trial. Health Expectations, 2004; 7: 617.
  • 60
    Dickson G, Green KL. Participatory action research: lessons learned with Aboriginal grandmothers.Health Care for Women International, 2001; 22: 471482.
  • 61
    Andejeski Y, Bisceglio IT, Dickersin K et al. Quantitative impact of including consumers in the scientific review of breast cancer research proposals. Journal of Women's Health & Gender-Based Medicine, 2002; 11: 379388.
  • 62
    Corneli AL, Piwoz EG, Bentley ME et al. UNC Project BAN Study Team. Involving communities in the design of clinical trial protocols: the BAN Study in Lilongwe, Malawi. Contemporary Clinical Trials, 2007; 28: 5967.
  • 63
    Ali KFC, Crome P. What patients want: consumer involvement in the design of a randomised controlled trial of routine oxygen supplementation after acute stroke. Stroke, 2006; 37: 865871.
  • 64
    Griffiths KM, Jorm AF, Christensen H. Academic consumer researchers: a bridge between consumers and researchers. Australian & New Zealand Journal of Psychiatry, 2004; 38: 191196.
  • 65
    Truman C, Raine P. Involving users in evaluation: the social relations of user participation in health research. Critical Public Health, 2001; 11: 215229.
  • 66
    Paterson C. ‘Take small steps to go a long way’ consumer involvement in research into complementary and alternative therapies. Complementary Therapies in Nursing Midwifery, 2004; 10: 150161.
  • 67
    Coupland H, Maher L, Enriquez J et al. Funding community-based participatory research: lessons learned. Journal of Interprofessional Care, 2004; 18: 428439.
  • 68
    Plumb M, Price W, Kavanaugh-Lynch M. Funding community-based participatory research: lessons learned. Journal of Interprofessional Care, 2004; 18: 428439.
  • 69
    Dobbs L, Moore C. Engaging communities in area-based regeneration: the role of participatory evaluation. Policy Studies, 2002; 23: 157171.
  • 70
    Angell K, Kreshka M, McCoy R et al. Psychosocial intervention for rural women with breast cancer: the Sierra-Stanford Partnership. Journal of General Internal Medicine, 2003; 18: 499507.
  • 71
    Meyer M, Torres S, Cermeno N, MacLean L, Monzon R. Immigrant women implementing participatory research in health promotion. Western Journal of Nursing Research, 2003; 25: 815834.
  • 72
    Elliott E, Watson A, Harries U. Harnessing expertise: involving peer interviewers in qualitative research with hard-to-reach populations. Health Expectations, 2002; 5: 172178.
  • 73
    Wright D, Corner J, Hopkinson J, Foster C. Listening to the views of people affected by cancer about cancer research: an example of participatory research in setting the cancer research agenda. Health Expectations, 2006; 9: 312.
  • 74
    Lucock M, Mirza M, Sharma I. Service users' views of a self help pack for anxiety. Journal of Mental Health, 2007; 16: 635646.
  • 75
    Ross F, Donovan S, Brearly S et al. Involving older people in research-methodological issues. Health Social Care Community, 2005; 13: 268275.
  • 76
    Rose D. Telling different stories: user involvement in mental health research. Research and Policy Planning, 2004; 22: 2330.
  • 77
    Philpot M, Collins C, Trivedi P, Treloar A, Gallacher S, Rose D. Eliciting users‟ views of ECT in two mental health trusts with a user-designed questionnaire. Journal of Mental health, 2004; 1394: 403413.
  • 78
    Andejeski Y, Breslau ES, Hart E et al. Benefits and drawbacks of including consumer reviewers in the scientific merit review of breast cancer research. Journal of Women's Health & Gender-Based Medicine, 2002; 11: 119136.
  • 79
    Ong BN, Hooper H. Involving users in low back pain research. Health Expectations, 2003; 6: 332341.
  • 80
    Cornes M, Peardon J, Manthorpe J. Wise owls and professors: the role of older researchers in the review of the National Service Framework for Older People. Health Expectations, 2008; 11: 409417.
  • 81
    Guarino P, Elbourne D, Carpenter J, Peduzzi P. Consumer involvement in consent document development: a multicenter cluster randomized trial to assess study participants' understanding. Clinical Trials, 2006; 3: 17407745.
  • 82
    Chambers R, O'Brien LM, Linnell S, Sharp S. Why don't health researchers report consumer involvement? Quality in Primary Care, 2004; 12: 151157.
  • 83
    Shea B, Santesso N, Qualman A et al. Cochrane Musculoskeletal Consumer GroupConsumer-driven health care: building partnerships in research. Health Expectations, 2005; 8: 352359.
  • 84
    Sutton J, Weiss M. Involving patients as advisers in pharmacy practice research: what are the benefits? International Journal of Pharmacy Practice, 2008; 16: 231238.
  • 85
    Mockford C, Staniszewska S, Griffiths F, Herron Marx S. The Impact of Patient and Public Involvement in the UK NHS Healthcare Service: A Systematic Review. Coventry: NHS Centre for Involvement, 2009.
  • 86
    Faulkner A. Capturing the Experiences of those Involved in the TRUE Project. A Story of Colliding Worlds. Eastleigh: Involve, 2004.
  • 87
    Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. International Journal of Health Technology Assessment, 2012; 27: 391399.