Patient and public involvement in medical performance processes: A systematic review

Abstract Background Patient and public involvement (PPI) continues to develop as a central policy agenda in health care. The patient voice is seen as relevant, informative and can drive service improvement. However, critical exploration of PPI's role within monitoring and informing medical performance processes remains limited. Objective To explore and evaluate the contribution of PPI in medical performance processes to understand its extent, purpose and process. Search strategy The electronic databases PubMed, PsycINFO and Google Scholar were systematically searched for studies published between 2004 and 2018. Inclusion criteria Studies involving doctors and patients and all forms of patient input (eg, patient feedback) associated with medical performance were included. Data extraction and synthesis Using an inductive approach to analysis and synthesis, a coding framework was developed which was structured around three key themes: issues that shape PPI in medical performance processes; mechanisms for PPI; and the potential impacts of PPI on medical performance processes. Main results From 4772 studies, 48 articles (from 10 countries) met the inclusion criteria. Findings suggest that the extent of PPI in medical performance processes globally is highly variable and is primarily achieved through providing patient feedback or complaints. The emerging evidence suggests that PPI can encourage improvements in the quality of patient care, enable professional development and promote professionalism. Discussion and conclusions Developing more innovative methods of PPI beyond patient feedback and complaints may help revolutionize the practice of PPI into a collaborative partnership, facilitating the development of proactive relationships between the medical profession, patients and the public.


| BACKG ROU N D
Internationally, patient and public involvement (PPI) in health care has been described as "central to the reform of Western economies" and its growth reflects the realization that the patient voice is relevant, informative and drives service improvement. [1][2][3] Whilst there is a developing academic literature base for PPI in health services, research and education, little is known of the evidence for PPI in the sphere of professional, and specifically medical, performance.
The last 20 years have witnessed a significant shift towards greater public accountability from health service organizations and health professionals, a possible consequence of which is the increased prominence of PPI. In the United Kingdom, the Health and Social Care Act (2001) introduced statutory PPI in service development, delivery and evaluation and is seen as a pivotal juncture in the evolution of PPI in health care-related research and education. 4,5 In the United States, the Hospital Consumer Assessment of Health Providers and Systems (HCAPHS) surveys were thought to have stimulated greater PPI in health care. However, Australia, New Zealand, Canada and most European countries (Norway and the Netherlands aside) have limited systems to capture and measure patient experience at a national level, although regional and local arrangements may exist. 6,7 However, in contrast with the developing evidence base for the impacts of PPI in health services, far less is known about the inclusion of PPI in medical performance processes and its impacts in this professional sphere. Globally, recertification, re-licensure and revalidation are terms that have been used to describe a process by which a doctor's performance is continually assessed, ensuring they are up to date and fit to practice, reassuring patients and the public that they remain competent throughout their careers. 8,9 Internationally, PPI in medical performance processes varies considerably. Several countries have appointed members of the public to licensing boards and professional associations, a trend borne from a greater societal and governmental desire for accountability from the medical profession. 10,11 Additionally, despite countries adopting recertification or re-licensure of doctors, 12 the PPI element in these processes is seldom reported in the academic literature. For example, in Belgium, evidence for continuing medical education (CME) involves a review of complaints or compliments. 13 The College of Physicians and Surgeons of Alberta, Canada, the statutory medical registration body for the province, has adopted a multi-source feedback (MSF) system for all physicians/surgeons in its jurisdiction. 14 15 Though examples of PPI within medical performance processes and regulation are evident internationally, much of the evidence in this domain originates from studies of PPI in medical revalidation in the UK. 16 In 2012, medical revalidation was mandated for all doctors in the UK. The Picker Institute's report, The Patient Voice in Revalidation, viewed revalidation as a necessary patient focussed reform, making patients its beneficiaries by representing them in some of its key tenets: "reassure the public," "ensure patient safety" and "public trust." 17 Whilst improved patient care is seen as the purpose of revalidation, PPI in the infrastructure, systems and processes of revalidation is currently limited to patient feedback on an individual doctor and disparate lay representation on local and national steering, advisory and implementation groups. 18 Individual doctors are required to submit and (reflect upon) patient feedback as part of their appraisal portfolio, once in their revalidation cycle (normally every 5 years). 19 A recent report evaluating medical revalidation in the UK found that overall, PPI in revalidation was viewed favourably by most stakeholders but there remained some confusion over its intended purpose and models of delivery. 20 Against this background, in this review we aimed to establish the contribution of PPI in medical performance processes internationally by exploring how PPI is operationalized, establishing the gateways and barriers to PPI in medical performance processes and understanding how PPI in all forms of patient input is influential in changing or modifying the practice of doctors.

| Design
The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines, 21

and Popay's
Guidance on the conduct of narrative synthesis in systematic reviews. 22 The review protocol is published on the PROSPERO website (registration number CRD42016035969). 23

| Eligibility criteria
There is large conceptual variation around the terms used to describe PPI. The terms "patients" and "public" are often used interchangeably as are "involvement," "participation" and "engagement." 24 This was considered when developing the search terms facilitated by the PICOS (population, intervention, comparator, outcome, study design) framework (Table 1). 25 We assessed studies against eligibility criteria based on the PICOS elements. For the "population," studies involving medical regulation stakeholders such as the public, patients and doctors, as well as the infrastructure for regulation, the national, regional (or federal) medical regulators or boards, professional bodies (eg, Royal colleges) and patient groups were included. In terms of the intervention, we included studies comprising all forms of patient input including lay representation, patient feedback, online reviews, information from patient surveys (experience/satisfaction), compliments and complaints. Through scoping, it was noted that no studies identified regulation as a specific outcome; hence, criteria were broadened to include outcomes relating to doctor/physician performance. Study design was not used as a basis for exclusion; however, we did exclude reviews, commentaries, opinion papers, etc., as well as studies associated with PPI in clinical decision making, research, education, health service provision or in the regulation/performance of other health professionals. Studies assessing the validity of patient feedback/satisfaction/experience tools were also excluded.

| Search: Study selection
Electronic databases MEDLINE, PsycINFO and Google Scholar were systematically searched for articles published in the English Language between January 2004 and June 2018. Although this review considers the role of PPI in medical performance globally, we selected January 2004 as a start date, as around this time there was growing discussion of the role of PPI in future proposals for revalidation in the UK. Electronic database searches were supplemented with ancestry and forward citation searches. Two independent reviewers undertook the review process at each stage. Duplicate studies were removed electronically and doublechecked by a second researcher. Studies were selected using a two-stage process. Firstly, all identified titles and abstracts were screened by each of the reviewers using previously agreed inclusion/exclusion criteria ( Table 1). Articles of included abstracts were then reviewed independently by each reviewer in full and assessed against the eligibility criteria. Discrepancies were resolved by discussion or sent to a third reviewer until consensus was achieved.

| Quality appraisal
An assessment of the quality of studies included in the review was undertaken to provide a comparative measure of study quality rather than for study exclusion, particularly as PPI as a singular intervention in medical performance processes is not consistently applied and given its relatively recent emergence, this review did not intend to evaluate its effectiveness. Nevertheless, to inform the robustness of the synthesis, quality assessment was undertaken using appropriate tools such as CASP for qualitative studies; an adapted version of a quality appraisal check list for case series studies; and "doctor" OR "physician" AND "patient involvement/engagement/participation/feedback/ experience/satisfaction/survey/service user/lay/co-production" OR "public involvement/engagement/participation" Types of studies All types of empirical studies (excluding reviews).
Methodological quality-not used as an exclusion criterion but considered when synthesizing the evidence for all studies.
Inclusion Population: Regulation stakeholders; public, patients and doctors as well as the infrastructure for regulation; the national, regional (or federal) medical regulators or boards, professional bodies (eg, Royal colleges) and patient groups. Intervention: All forms of patient input: lay representation, PPGs, patient feedback, online reviews, information from patient surveys (experience/satisfaction), compliments and complaints. Outcome: Studies with an outcome linked to regulation or performance.

| Data extraction, analysis and synthesis
Data extracted from eligible studies were organized by the first reviewer under the following headings: year of publication, country in which study was undertaken, population (eg, patients/doctors), intervention (eg, complaints), context, study design, summary of findings and key themes (see Table S1).

| RE SULTS
The search identified 3638 articles (once duplicates had been removed). The titles and abstracts of these were screened and 87 were initially found to be relevant and full text versions were obtained.
Following full text assessment and preliminary synthesis, 37 studies were excluded based on their outcome, not related to performance, leaving 48 studies that met the eligibility criteria ( Figure 1). The key features of the included studies (categorized by study design, eg, cross-sectional study) including publication title, year of publication, author, country in which the study was undertaken, type of PPI intervention (eg, complaints) and quality appraisal score are summarized in Table 2.

| Quality appraisal
Quality appraisal of the included studies in this review was challenging for two reasons. Firstly, the heterogeneity of study designs used in the included studies limited comparison of study quality between studies.
Secondly, the quality appraisal tools did not exist in an original format and either required adaptation or were not directly relevant for the studies they were designed to assess, for example CASP for qualitative studies when applied to content analysis of free text responses from surveys. Hence, we did not use quality appraisal results to draw any conclusions on the overall findings from this review. Quality appraisal scores are listed in Table 2.
A coding framework drawn from the data in the included studies was produced and primarily arranged into three overarching themes issues shaping PPI, mechanisms for PPI and impact of PPI on the systems and processes of medical regulation. Within these themes, emerging sub-themes are presented with potential barriers and gateways for wider evolution or implementation of PPI models, based on the evidence for their positive and negative impacts, providing a narrative for PPI in different settings.

| Issues shaping patient and public involvement
The review has identified four main issues that shape PPI in medical

Attitudes of the doctor and profession
In some studies, the negative attitudes of doctors and the profession emerged as an important barrier, potentially hindering PPI from developing within systems and processes relating to medical performance. 29

Patient characteristics
Patient characteristics may act as barriers, limiting patient access to feedback or complaints systems. For example, tools for patient feedback were deemed inappropriate for certain age groups, for example children, 34 and access to and utilization of complaints systems were dependent upon age (older patients), socioeconomic status (low income) and ethnicity (minorities), with fewer complaints received from these groups, a specific concern raised from a study conducted in Australia and New Zealand.
The relatively low propensity to complain among patients who are elderly, socioeconomically deprived, or of Pacific ethnicity suggests troubling disparities in access to and utilisation of complaints processes. Further research is required to better understand and address these disparities. 35

Perceptions of purpose of complaints and feedback
There appears to be divergence between patients and doctors, and among doctors as a group on the purpose of complaints and feedback. The differing conceptualization of this purpose is a potential barrier to developing PPI in medical performance processes. For example, one study cited ambiguity relating to the purpose of patient feedback; patients were unsure as to whether they were providing feedback on the service or the individual doctor. 33 Some studies suggested that the purpose of complaints was to increase accountability and enhance professionalism in doctors. [36][37][38] In contrast, some doctors and patients perceived complaints as a punitive measure that highlighted issues with performance or competency. 39 In one study, patients suggested that disciplinary action against the doctor was not always the preferred outcome of lodging a complaint but, because complaints' systems were perceived as inadequate and unable to provide the reassurance that

| Mechanisms for patient and public involvement
Patient feedback was identified as a key mechanism for PPI in medical performance processes, especially in the UK. For doctors, the effectiveness of patient feedback tools was an important factor in the perceived value of the data obtained. This was associated with the validity of the tool and the reliability of the resulting data. Doctors in these studies suggested that patient feedback as part of MSF was a useful tool for formative improvement but queried the credibility of the data for performance or competency assessment. 33,41,44 Concerns related to the internal validity of the tools including bias in selection of patients by a doctor (or members of staff) and in responses received from patients skewed towards providing more favourable feedback.
Furthermore, the authors in these studies suggested that patient feedback scores did not always correlate with colleague feedback scores.
… concerns relating to aspects of methodology such as whether patients and colleagues can provide objective feedback may undermine its credibility as a tool for identifying poor performance. 44 Although colleagues appear to report poor performance using MSF, patients fail to report concurrent findings.
This challenges the validity of patient feedback as it is currently constructed. 45

Quality improvement
A positive outcome of complaints and negative feedback data was the opportunity for quality improvement for both the individual doctor and the service through learning from previous issues, testing new ideas and implementing different approaches to limit future problems. [48][49][50][51] Many authors also perceived complaints as a conduit for managing "at risk" doctors, enabling organizations to mitigate risk through performance management. 52,53 Complaints and patient satisfaction data have been previously proposed as a useful quality improvement tool. 54 Additionally, one study suggested that low patient satisfaction scores were a predictor for future complaints providing an opportunity to performance manage a doctor whilst enabling patients to participate in quality improvement.

| D ISCUSS I ON
This study has provided a systematic review and narrative synthesis of the international literatures on PPI in medical performance processes. The review has shown that PPI in medical performance processes is primarily through complaints and patient feedback with minimal patient input into the actual mechanisms. The review has produced a robust body of evidence identifying key gaps in the academic literature relating to PPI in medical performance processes in terms of (a) the extent of PPI, (b) shaping of the PPI agenda and (c) the impact of PPI on systems and processes.
In terms of shaping PPI in medical performance processes, a significant barrier identified was the doctor/profession attitude towards PPI. Whilst, for example, the General Medical Council in the UK established lay involvement at the uppermost levels of the organization well over a decade ago, PPI in regulatory processes is still largely through patient feedback. This review has shown that there is a need to establish the extent of PPI in medical performance processes. This is to ensure that the patient voice in the infrastructure and mechanisms of medical performance processes develop beyond lodging a complaint and completing a patient feedback or satisfaction form.
The focus for PPI has been described as being directed to regulatory strategy acting on the doctor/regulator relationship, rather than the doctor/patient relationship. 58 However, this review has found a growing discourse about the role of patient input in the doctor/patient relationship. At this interface, complaints and feedback data are thought to initiate changes in practice by the individual doctor, both positive (quality improvement) and negative (defensive practice). 30,32,59 Unintended and negative consequences such as defensive practice or the impact on a doctor's self-confidence are potential risks to the quality of patient care. 29,47 Nonetheless, some within the profession acknowledge that patients have a role to play in complaints procedures. 29 Addressing negative attitudes is challenging and reflects the current conceptualization of PPI in health care whereby some health professionals and organizations struggle to embrace the notion of partnership with patients and even feel threatened by the idea of active involvement, favouring consultation over collaboration. 60 The review has shown that doctors view feedback and complaints as both a summative and formative assessment of their performance.
In the included studies, doctors were particularly concerned about feedback and complaints data being used for summative assessment and in a minority of cases, doctors perceived complaints as a potentially punitive measure. If feedback and complaints were perceived as having a formative function, they may be viewed more favourably and the patient's view held in higher regard. In Alberta, Canada, patient feedback used for the purposes of recertification is mandated but data cannot be subpoenaed in a court of law and thus mitigates the perception that such data will be used for litigation purposes. 44 Better advocacy of the purpose of complaints and feedback for doctors and patients may provide more meaningful insights for a doc- Indeed, in other spheres of health care, PPI in quality improvement has been suggested as positively influencing organizational culture by increasing emphasis on non-hierarchical, multidisciplinary collaboration, encouraging staff to model desired behaviours of recognition and respect, and commitment to rapid translation of research into practice. 62 The barriers to PPI in medical performance processes identified in the review could also be viewed as opportunities. The existence of complaint systems in numerous countries is promising and provides a mechanism by which patients can participate in the assessment of a doctor's performance. Furthermore, the recognition of limited accessibility to feedback and complaints systems for certain demographic groups is also encouraging providing organizations and patient groups with "targets" for their advocacy.
Older patients and those from certain ethnic backgrounds are less likely to lodge complaints or provide feedback on their doctor. 35 Understanding the reasons for this is required to better engage these groups in PPI. Innovative approaches to patient feedback collection such as the use of touchscreens at the point of service may improve response rates as they are accessible and inclusive to most. 51 The potential positive impacts of PPI outlined in this review such as promoting professionalism among doctors and improving the quality of care delivery require a greater focus in future research studies. Authors in some of the included studies focused on the reasons for complaints and feedback being less impactful, citing tools and data as limitations. This was exemplified by concerns of the credibility of patient feedback data with some doctors critiquing the design of tools, questioning the process of collecting data (selection bias) and the reliability of responses received from patients (response bias). 45,63,64 This is despite tools having been repeatedly tested for their validity and generalizability, with reasonable evidence to suggest that they are reliable. 63,65 The review has identified the need for a better understanding of the actual impact of the different types of PPI in their current format in regulatory processes and systems, at the level at which patients participate in medical performance processes, that is through complaints and feedback both of which may indicate clinical, managerial and broader systemic issues or a deterioration in the doctor-patient (or service-patient) relationship. 66 However, in this review complaints and negative feedback have been identified as possible conduits for individual doctor and service improvement. Thus, PPI has a potentially significant role in improving the quality, relevance and ultimately the value of complaint and feedback mechanisms, which is integral to promoting accountability and professionalism, thus enhancing the doctorpatient relationship.

| Strengths and limitations of this review
The uncertainty of the precise definition of medical performance somewhat hindered the assessment of studies for eligibility in this review; yet, the included studies focused on aspects closely associated with medical performance such as professionalism, competency and professional development. Even so, without a precise definition for medical performance it is possible some studies were missed. In an attempt to overcome this issue, the review encompassed the international literature on PPI in medical performance processes, including studies from several countries with different medical regulatory systems and approaches for assessing medical performance within which the extent of PPI was somewhat varied. Nonetheless, given the heterogeneity of contexts and systems it is challenging for this review to provide standardized recommendations for developing PPI in medical performance processes.
This study has used a robust approach to review the evidence for PPI in medical performance processes including a quality appraisal of included studies. Additionally, the use of a narrative synthesis is important as it has provided the opportunity to use words and text to summarize and explain findings from the reviewed literature thus providing evidence on the barriers and gateways to PPI in medical performance processes whilst highlighting the key evidence gaps that need to be addressed.

| CON CLUS ION
The significance and recognition of PPI have grown in many domains of health care in recent years propagating an evolution of "patientcentred care" and shared clinical decision making. This review indicates a need for a similar level of integration for PPI within medical performance processes as existing models are both fragmented and inadequate to have a meaningful impact on systems and processes that assess and monitor performance. More broadly, quality improvement may act as a driver for PPI in medical performance processes to evolve beyond the level of providing feedback and lodging complaints, forming the foundation of a transition from a culture of contractual PPI that exists as part of the clinical interface between the doctor and patient, to that of collaboration that enhances the profession-society relationship.

ACK N OWLED G EM ENTS
We would like to acknowledge the contribution of Suzanne Nunn and Nicola Brennan who provided advice and guidance during the scoping phase of the review. We would also like to acknowledge members of the UMbRELLA collaboration and in particular Alan Boyd for providing input and insights on the review content.