Perceptions of people with respiratory problems on physician performance evaluation—A qualitative study

Abstract Background Despite increasing calls for patient and public involvement in health‐care quality improvement, the question of how patient evaluations can contribute to physician learning and performance assessment has received scant attention. Objective The objective of this study was to explore, amid calls for patient involvement in quality assurance, patients' perspectives on their role in the evaluation of physician performance and to support physicians’ learning and decision making on professional competence. Design A qualitative study based on semi‐structured interviews. Setting and Participants The study took place in a secondary care setting in the Netherlands. The authors selected 25 patients from two Dutch hospitals and through the Dutch Lung Foundation, using purposive sampling. Methods Data were analysed according to the principles of template analysis, based on an a priori coding framework developed from the literature about patient empowerment, feedback and performance assessment. Results The analysis unearthed three predominant patient perspectives: the proactive perspective, the restrained perspective and the outsider perspective. These perspectives differed in terms of perceived power dynamics within the doctor‐patient relationship, patients' perceived ability, and willingness to provide feedback and evaluate their physician's performance. Patients' perspectives thus affected the role patients envisaged for themselves in evaluating physician performance. Discussion and conclusion Although not all patients are equally suitable or willing to be involved, patients can play a role in evaluating physician performance and continuing training through formative approaches. To involve patients successfully, it is imperative to distinguish between different patient perspectives and empower patients by ensuring a safe environment for feedback.


| INTRODUC TI ON
While patient empowerment is gaining momentum, 1,2 the involvement of patients, hereinafter referred to as patient and public involvement (PPI), in the improvement of health-care quality, particularly in the evaluation of health-care professional performance, is often lacking or underreported. [3][4][5][6][7][8][9] PPI across the medical education continuum ranges from patients' participation in teaching, feedback and assessment or involvement in course design towards partnership and collaboration. 10 Lalani and colleagues present the worldwide variability of PPI across medical performance processes and call for more collaborative ways of involvement, beyond formal patient feedback and complaints. 11 Patients are the very essence of why healthcare systems exist and as health-care consumers, they have a direct stake in the way both quality and providers of care are evaluated. 12,13 Patients' participation in feedback processes as a form of PPI is generally established through evaluation or satisfaction surveys in which patients communicate their views on care received or evaluate health-care processes and physicians' professional practice. 4,11,14 Research findings show that the inclusion of patients' views renders performance evaluations more holistic and transparent, potentially allowing physicians to reflect on their practice. [15][16][17] Although different in focus from feedback received from peers or other health-care workers, patient feedback can provide physicians with valuable information on how to improve their learning and performance. 10 Similarly, patients' evaluations of physician performance can help to make decisions about physician competence and to identify underperforming physicians. 18 Especially in the assessment of non-clinical competences, such as communication and professionalism, patient evaluations on physicians' performance can serve as meaningful additional evidence. 11,[19][20][21][22] Patient involvement in physician performance evaluation, however, reaches further than restrictive satisfaction questionnaires. It also entails lay representation in the design of performance evaluation processes or guideline development and strategic planning. 10 How patient evaluations can contribute to physicians' learning and performance assessment has received little attention-particularly informal patient feedback on the individual functioning of a physician. 23 Providing feedback is a complex cognitive and affective process and the resulting evaluation is determined by its provider's beliefs, cognition and emotions. 24 Even though patients' beliefs are often presumed to be known, a recent review on the impact of patient feedback of physician's performance highlights that research on patient feedback from the patient perspective is currently lacking. 25 Although Lalani et al 11 disclose that patient characteristics such as age or socio-economic characteristics may act as barriers to PPI, the authors do not discuss the underlying processes or address patients' perspectives on their role in physician performance evaluation or evaluation systems. In order to understand how to use patient-generated data, however, we need to explore the assessor perspective and likewise add to a clear conceptual understanding of 'the patient perspective'. 26,27 Before we can address such a practical need, we must unpick factors that influence patients' possible role in physicians' learning and performance evaluation, such as their beliefs, preferences and concerns. 10,28,29 By addressing these gaps, we may be able to achieve a meaningful patient contribution to the evaluation of physician performance processes. 30,31 The purpose of the present study, therefore, is to answer the question: What are patients' perspectives on their possible role within the evaluation of physician performance and physicians' lifelong learning, particularly in providing feedback?

| ME THODS
We conducted a qualitative study based on semi-structured interviews with the aim to explore patients' notions of evaluation of physician performance, and to better understand their perspectives on their role in the evaluation of physician performance by providing feedback.

| Setting and participants
The study was set in The Netherlands that has an obligatory national recertification system in place with limited PPI (Box1). data; or in the preparation or approval of the manuscript or the decision to submit for publication. All authors had full access to the data and can take responsibility for the integrity of the data and the accuracy of the data analysis.

K E Y W O R D S
empowerment, feedback, lifelong learning, patient and public involvement, patient perspective, performance evaluation, power dynamics, recertification, revalidation, voice

Box 1 The Dutch recertification system
The Dutch recertification system emphasises continuing development over the detection of malpractice. At present, medical specialists must meet the following three requirements after each period of 5 years: (a) they must prove that they have practised medicine sufficiently and regularly (ie ≥ 16 hours per week on average); (b) they must have engaged in continuing medical education (CME) activities worth 200 CME points; and (c) they must have undergone an external quality assessment of their department by a committee of the National Specialty Society.
As of 2020, an additional requirement will apply: (d) physicians must demonstrate that they did prepare a personal development plan and participated in an assessment of individual functioning.
We selected patients using purposive sampling based on the following inclusion and exclusion criteria. First, we aimed to include patients who were most likely to have developed a long-standing or intensive treatment relationship with their physician and who had high levels of experience regarding health-care delivery. Therefore, we decided to include patients with lung cancer or a chronic lung disease. Second, in order to obtain maximum variation of the patient population, we selected patients with a variation in burden of disease and age. Third, we wanted the sample to reflect varying degrees of patient experiences, views and knowledge, and therefore included not only individual patients, but also patient group members and patient representatives. 32 Finally, we excluded patients who practised as physicians themselves or who were receiving care from any of the researchers at the time of the study or in the past.
We approached patients in two ways: First, we asked the Dutch Lung Foundation to include a call in their periodical newsletter to their patient panel, inviting interested individuals to contact the first author through the foundation. This resulted in enrolling ten patients. Second, to sample across different diseases within secondary care, we visited the respiratory outpatient clinic of one academic and one non-academic hospital, which yielded seven and three patients, respectively. To include not only patients, but also their informal carers (mostly partners), 33 we also enrolled five patient partners, leading to a total number of 25 participants (12 male and 13 female).
The mean age of participants enrolled was 65 (ranging from 35 to 82 years old, SD = 10.9).

| Data collection
We developed the interview protocol based on literature on patient engagement, evaluation of physician performance and feedback for performance assessment purposes in the health professions. We used the literature 12,14,20 to include questions that asked explicitly about whether and how patients envisaged a role for themselves in providing informal feedback and evaluating physicians beyond formal satisfaction questionnaires. Having piloted the interview guide (Appendix S1) by conducting the interview with patients who were not included in the study, we revised and simplified the language of the introductory questions. CS interviewed members of the Lung Foundation via phone, while outpatients were interviewed either face-to-face after their visit to the clinic or by phone. Semi-structured interviews lasted 37 minutes on average (SD = 8.1) and were transcribed verbatim. We collected and analysed the data in an iterative process, allowing the analysis to inform subsequent interviews.
Data collection and simultaneous analysis took place from June to August 2018, until the research team agreed that thematic saturation was reached. 34

| Patient involvement in this study
Besides enrolling patients as research participants, a patient, TGJT, was also a member of the research team and co-author. Being a chronic patient herself who has extensively researched the topic of PPI, TGJT represented the patient voice in the research team by advising on the feasibility and burden of ideas and pointing out potential pitfalls in the study design and conduct. After publication of this study, the results will be distributed in a plain language summary to the research participants and wider patient groups.

| Data analysis
We performed a template analysis of our data, which is a form of thematic analysis. 35 In accordance with this technique, we iteratively applied a sequence of templates to the data set, starting with a priori codes followed by constant modification of themes throughout the analysis. As a first step, the primary researcher CS familiarized herself with the data and initially coded five interview transcripts based on an a priori coding framework that was developed from the literature about PPI, feedback and performance assessment. A priori codes based on the literature 21,28,36,37 included perceptions of the doctor-patient relationship and communication, particularly in light of a potential hierarchical relationship, the role of feedback, and in particular the preferred way of providing feedback, patient empowerment, PPI and patient identity. As a second step, CS modified and replenished the initial codes during the analysis of further interviews, which led to an initial template. This template served to describe whether patients envisaged a role for themselves in the evaluation of physician performance and depicted the levels of trust patients experienced in the relationship with their physician and associated feelings and readiness to provide feedback. As a third step, CS and SM discussed themes; and devised and produced a final template that included themes around patient voices and power dynamics. Based on this final template, CS and FWJMS independently coded and discussed two more transcripts. CS, SM and FWJMS subsequently discussed preliminary interpretations, following which they refined the final template into a focused template (Appendix S2). CS applied this focused template to all interview transcripts and discussed the findings with the entire research team until they reached consensus about the final interpretation. 35 We ensured validity by conducting a member check among interviewees who confirmed our interpretations. Although all participants were offered the opportunity to participate in the member check, only one participant responded, which had no consequences for data interpretation. We used the software programme ATLAS.ti to manage our data, and the COREQ checklist to report on analysis (Appendix S3). 38

| Reflexivity
In order to maintain the quality of the study, reflective memos were used throughout data collection. CS has a background in health sciences and is a PhD Student in medical education. TGJT conducted scientific research in the area of patient involvement and is also a patient expert and volunteer at the Dutch Lung Foundation. She conjured up the patient's perspective in the study design and analysis. MJBG and EWD are both medical educators. SM is an education manager. Two research team members (FWJMS and GGUR) are respiratory physicians.
All members of the research team are involved in medical education, with either a special focus on assessment, continuing education and recertification or patient involvement. Together, the expertise of the research team members contributed to how we incorporated the patient voice into the evaluation of physician performance, on the one hand, while helping us critique its effect on performance evaluation.

| RE SULTS
Our analysis led to the construction of three recurring perspectives on the role patients envisaged for themselves in providing feedback and evaluating physicians' performance. In the following sections, we describe which distinct patient perspectives we encountered and how they are characterized.
The predominant perspectives were shaped by patients' personal experiences and the consequences they expected to follow from evaluating their physician. The extent to which patients experienced a power balance within the doctor-patient relationship seemed to affect the role they envisaged for themselves. The perceived power dynamics of the doctor-patient relationship affected patients' perspective on their role in the evaluation of physician performance and their willingness to provide feedback. By levels of power, we refer to patients' perceived dependency on their attending physician during treatment. Table 1 summarizes the three predominant perspectives.
It should be borne in mind that these three overarching, predominant patient perspectives by no means detract from the fact that each patient is unique. Even though we illustrate characteristics of three perspectives, every patient interviewed had their own individual feeling on their perceived 'place' to offer feedback, and to decide whether they followed through with this. Some interviewees exhibited characteristics of more than one perspective, or were doubtful of their role in physicians' continuing learning and performance evaluation and shifted between different perspectives. That is, the perspectives presented are not fixed categories but should rather be seen as a continuum across which patients can move, depending on time and context. Regardless of the different perspectives, most interviewees recognized the importance of physicians' continuing learning and performance evaluation systems: 'continuing training is really important because otherwise you will be overtaken by events at some point' (Interview 7). Yet, they envisaged their involvement mainly as providing feedback and not as being involved as a lay representative in system design, although some clearly expressed feelings about the need to feel safe within the system. In regard to their role in providing feedback, our interviewees voiced clear ideas about which physician competencies they were able to evaluate.
They mostly addressed professionalism and communication as well as collaboration skills.

| The proactive perspective
Patients who shared this perspective were assertive and had a relationship with their physician in which they felt power was equally TA B L E 1 Distinct patient voices in the evaluation of physician performance: what are the predominant perspectives?

| The restrained perspective
Patients who showed characteristics of the restrained perspective did not envisage an active role for themselves in the evaluation of physician performance. These patients trusted that the current system would assure physicians' competence and quality of care. Consequently, they were reserved in offering their opinion: 'Well, that is not necessary. I know that they regulate it from above through, through an organization, or the government… I think that is sufficient' (Interview 8). Considering their views as subordinate to their physicians, these patients did not spontaneously provide feedback or feel a need to evaluate their physician, especially not when they had complaints because 'as patient you just do not dare to' (Interview 8). They fully relied on their physicians' competence and therefore did not venture to question them. As one interviewee explained: 'Because you also assume so.

| D ISCUSS I ON
With this study, we aimed to explore patients' perspectives on their role within the evaluation of physician performance. We were able to define three predominant patient perspectives that depended on the extent to which patients felt competent to take this role and to which they experienced a power balance within the doctor-patient relationship: the proactive perspective, the restrained perspective and the outsider perspective.
Reflecting on the challenges inherent in PPI, 31,39 our results underline that there is no such thing as a 'collective' patient voice, but that a multitude of patient perspectives must be considered.
Indeed, not only are patient perspectives individually bound, they are tied to a specific moment in time. 26 and hierarchy as well as a lack of common language between patients and physicians. 42 The latter argument, however, might be invalid for patient experts or representatives, who are well trained to discuss patient perspectives with professionals and policy makers. 32 Consequently, physicians might more readily accept feedback from these patients experts, presuming they have an understanding of their medical work. 43 This argues for more effort in the field of patient education and improved power dynamics, and suggests a change in the future once reliable patient-generated information become increasingly available.
Altogether, this highlights that not only the provision of feedback, but also its acceptance can be challenging for patients and physicians, respectively. The type of feedback, its credibility and the competence addressed, determines whether physicians accept patient feedback. 43

| Strengths and weaknesses
First, the main strength of our work is the rigour with which we performed the data analysis, characterized by the iterative analysis process.

ACK N OWLED G EM ENTS
We wish to thank all patients for their participation in our research. We also acknowledge the Dutch Lung Foundation, the

MUMC + Department of Respiratory Medicine and the Catharina
Hospital in Eindhoven for their immense help in contacting potential study participants. We also thank Angelique van den Heuvel for her language review.

CO N FLI C T O F I NTE R E S T
The authors declare no competing interests other than the funding listed above. Moreover, all authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare: no support from any organization for the submitted work other than reported; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work.
AU TH O R S ' CO NTR IB U TI O N S All authors designed and conceived the study. CS wrote the research plan, collected and analysed the data and drafted and revised the paper. She is guarantor. MJBG analysed the data, and drafted and revised the paper. SM drafted and revised the paper.
TGJT analysed the data, and drafted and revised the paper. FWJMS analysed the data, and drafted and revised the paper. ED drafted and revised the paper. GGUR drafted and revised the paper. All authors engaged in the final steps of analysing the data. As corresponding author, [first author] attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

E TH I C A L A PPROVA L
We informed potential study participants about the research as soon as they contacted the Dutch Lung Foundation or registered for their consultation at the outpatient clinic. We emphasized that the research was entirely independent of the care received and that the treating physician would not receive any information about the interview. All participants gave written and oral informed consent.
We obtained ethics approval from both the Netherlands

TR A N S PA R EN C Y S TATEM ENT
The lead author and guarantor of this manuscript affirms that this manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as originally The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.