What's the problem with patient experience feedback? A macro and micro understanding, based on findings from a three‐site UK qualitative study

Abstract Context Collecting feedback from patients about their experiences of health care is an important activity. However, improvement based on this feedback rarely materializes. In this study, we focus on answering the question—“what is impeding the use of patient experience feedback?” Methods We conducted a qualitative study in 2016 across three NHS hospital Trusts in the North of England. Focus groups were undertaken with ward‐based staff, and hospital managers were interviewed in‐depth (50 participants). We conducted a conceptual‐level analysis. Findings On a macro level, we found that the intense focus on the collection of patient experience feedback has developed into its own self‐perpetuating industry with a significant allocation of resource, effort and time being expended on this task. This is often at the expense of pan‐organizational learning or improvements being made. On a micro level, ward staff struggled to interact with feedback due to its complexity with questions raised about the value, validity and timeliness of data sources. Conclusions Macro and micro prohibiting factors come together in a perfect storm which provides a substantial impediment to improvements being made. Recommendations for policy change are put forward alongside recognition that high‐level organizational culture/systems are currently too sluggish to allow fruitful learning and action to occur from the feedback that patients give.


| INTRODUC TI ON
The patient experience agenda is reaching a zeitgeist moment in many health-care systems globally. Patients are increasingly giving feedback on their experiences of health care via a myriad of different methods and technologies. Most commonly, these take the form of national surveys, formal complaints and compliments and social media outlets. Various publications outline a range and diversity of qualitative methods for gaining rich feedback from patients. 1 Several systematic reviews have identified a range of quantitative survey tools which are used across the world to capture patient experience in an inpatient setting. 2,3 These include large-scale surveys such as the NHS National Inpatient Survey in the UK and the Hospital Consumer Assessment of Healthcare Providers and Systems in the United States. 3 Currently, in the UK, major resource is being given to the collection of the Friends and Family Test 4 which has been mandatory since 2014 for all acute hospital Trusts to collect. A significant driving force for the current focus on gathering patient experience feedback in the UK arose from national level recommendations such as the Francis and Keogh reports. 5,6 Internationally, "Better Together" in the United States and "Partnering with Consumers" in Australia demonstrate that this focus has been mirrored internationally. 7 It is now widely acknowledged that patients want to give feedback about health care 8 and recommended that staff listen to what their patients say about the experience of being in hospital. 9 Yet, whether staff can use this feedback to make changes to improve the experiences that patients have is now a central concern. [10][11][12][13][14][15] This pertains to differing areas of the health-care system from senior management at the level of the hospital board (formalized group of directors) down to individual clinicians working on the frontline. Hospital boards have received recent attention to understand the ways in which they use patient feedback to improve care at a strategic level 16 and how they govern for quality improvement. 17 There is a concern that the ever growing collection of feedback is not being used for improvement but, rather, represents a "tick box mentality" of organizations thinking they are listening to their patients views but actually not doing so. 18 Recent work in the UK has looked at how health-care professionals make sense of why patients and families make complaints about elements of their care 19 and found that it was rare for complaints to be used as grounds for making improvements.
Almost everyone interested in health-care improvement, and certainly those providing frontline care now have a vested interest in listening to patients. 15 However, a myriad of challenges are still preventing the wide-scale effective use of patient feedback data for quality improvement. This can be contrasted against a backdrop of a simultaneous "movement for improvement" 20 where grassroots, bottom-up approaches to health-care improvement are being championed. It is interesting to note that, despite the recent paradigm shift in the literature which acknowledges this "patient feedback chasm", 21 most commentators have so far only paid attention to the problems at the micro level.
Flott et al 11 discuss problems related to data quality, interpretation and analytic complexity of feedback and then put forward ideas for how the data itself could be improved to allow staff to engage with it better. Likewise, Gleeson et al 12 found a lack of expertise amongst staff to interpret feedback and issues surrounding the timeliness of it, coupled with a lack of time to act on the data received. Sheard et al 15 explored why ward staff find it difficult to make changes based on patient feedback. They found that effective change largely relates to an individual or small teams structural legitimacy within the health-care system and that high-level systems often unintentionally hindered meso and macro level improvements which staff wished to make.
In this study, we report the findings from a qualitative study undertaken at three hospital Trusts in the North of England. We were interested in which types of patient experience data were being collected, how staff were or were not using this data and whether there was a relationship to improvement on the wards.
Here, we base our reporting on the question "what is impeding the use of patient experience feedback?" which is examined through both a macro and micro lens. We concentrate on this finding as it arose from the participants as being of central importance. It is important to define what we mean by the use of the terms "macro" and "micro" within this study. Here, macro refers to the system, organization, structure or strategy, for instance, the hospital culture, how teams or processes are set up or ways of working. Micro refers to the issues with sources of feedback and how individuals use or interact with them.

| Setting
We conducted a mixed-method qualitative study using focus groups and interviews across three NHS hospital Trusts in the North of England. This qualitative study was the first work package in a programme of research; whereby, the overall purpose was to develop a patient experience improvement toolkit to assist ward staff to make better use of patient experience feedback. The three Trusts were selected to provide diversity in size and patient population. Then, two wards per Trust were approached to take part in the study leading to six wards working with us. We sampled the six wards based on a divergence of speciality, size and patient throughput. The specialities of the wards were as follows: accident and emergency department, male surgery (this represents two wards at different Trusts), maternity department (including ante-and postnatal services), female general medicine and an intermediate care ward for older patients.

| Sampling
Ward staff mostly represented opportunistic sampling, and management participants were sampled for maximum variation. Ward staff predominantly encompassed senior and junior nursing staff, support workers and the inclusion of allied health professionals in some of the focus groups. Management participants were drawn from a range of roles occupying middle-and senior-level hospital management such as patient experience managers or heads of patient experience, matrons, heads of nursing (and their deputies), research leads, medical, quality, risk, governance and performance directors.
The bulk of interview participants worked directly in or managed patient experience teams.

| Datacollection
Fieldwork took place between February and August 2016.

University of Leeds ethical approval was secured in October 2015
(ref: , and NHS Health Research Authority governance approval was granted in February 2016. All participants gave written, informed consent. Ward staff took part in focus groups, and management staff took part in individual in-depth interviews. Seven focus groups and 23 individual interviews were conducted. Focus groups ranged from three to seven participants, and two management participants were interviewed as a dyad. The average length of an interview was 55 minutes and 45 minutes for a focus group.
In total, 50 participants took part in this qualitative study. All focus groups and interviews were conducted face to face in staff offices, digitally recorded and then transcribed by a professional transcriber.
RP collected all interview data. LS, RP and CM all collected focus group data. All are experienced qualitative health researchers with doctorates in their respective fields.

| Topicguidequestions
Two topic guides were devised; one for the data collection from ward staff and another for management participants. Headline topic guide questioning was derived from the literature. [1][2][3]9,14 Focus group questioning centred on what types of patient experience feedback the participants received, how they engaged with it and responded to it and where/how it fitted in with their everyday clinical work.
Interview questioning explored the different kinds of patient experience feedback available to the Trust and how these were generated, prioritized and managed at the level of the ward, directorate and whole organization. The formats of the topic guides and that of interview questioning were flexible to allow participants to voice what they considered to be important. Both topic guides were piloted, with changes being made to the content and structure based on how initial participants responded to the interview or focus group questions. We included all data in the analysis and did not discard meaningful data which were gathered during piloting.

| Analysis
LS and RP took the same five interview transcripts, and each independently developed a provisional descriptive coding framework.
These five transcripts were chosen as those which were representative of the whole interview data set in terms of spread across the Trusts and general content. The same exercise was repeated for the focus groups, albeit with three transcripts. LS and RP held an intense analysis session where they met (along with RL) to discuss the differences and similarities in their coding frameworks, although there was general parity amongst them. LS then returned to the selected transcripts and immersed herself in the data in order to devise an overall meta coding framework which would allow for data from both the interviews and focus groups to be coded together. This meta coding framework sought out themes on a conceptual level rather than a descriptive level. That is, rather than simply describing what the participants discussed, LS looked for the differing ways in which patient experience feedback was approached conceptually across the participants involved in both methods. Differences and similarities were identified with LS noticing that participants discussed the topic at different levels with the management interviewees tending to view patient experience feedback in a macro way (both explicitly and implicitly) and the ward staff focus group participants viewing it in a micro manner. The meta level coding framework was checked with RP for representativeness and accuracy. After slight modification, LS then coded all transcripts and some subthemes were modified as coding progressed. LS conducted further interpretive work to write-up the findings. Initially, we began by conducting a classic thematic analysis 22 but realized that this was not sufficient for our needs as thematic analysis often relies on portraying a descriptive account of participants' narratives. Instead, we looked to generate high-level conceptualizations from the data. The analysis was wholly inductive, and, as such, we did not structure it on any existing theoretical frameworks.

| FINDING S
Here, we briefly set the scene by describing the main sources of patient experience feedback in the UK before moving on to focus entirely on: "what is impeding the effective use of patient experience feedback?" All participants have been ascribed a number and a generalized descriptor of their role, rather than their precise role, to protect their identity. We will discuss two distinct groups of participants which we will call "ward staff" and "managers. or the environment to the clinician caring for them or to a more senior staff member. Conversely, it also included spontaneous thanks or praise given in an interpersonal exchange. In this study, we focus on formalized sources of patient experience feedback and discuss factors surrounding their effective use, as per our key areas of interest and research brief. However, it should be acknowledged that informal feedback was often used by ward staff in a timely way to improve the experience for the needs of a particular patient.

| Whatisimpedingtheeffectiveuseofpatient experience feedback?
We chose to focus on the factors that are impeding the use of feedback rather than an account which paid equal attention to the factors that were assisting. Whilst there were certainly instances where individual personnel and small teams had instigated processes and ways of working which were beneficial, these accounts were localized and not of sufficient importance to most participants about the topic at hand. Furthermore, attempts to improve issues identified in feedback sometimes led to unintended consequences which further problematized an already complex and fraught task.

| Atthemacrolevelofthehealthcare organization
Considering the data set as a whole, possibly the most striking element is the overwhelming nature of the industry of patient experience feedback. Ward staff at one hospital department at Trust C At each of the three hospital sites, a significant, system-wide level of resource, effort and time was being expended which primarily focused on maintaining the collection rates of feedback. This was coupled with layers of hierarchies and bureaucratic processes surrounding data collection which were said to be to be confusing to staff and patients alike. Mirroring the current NHS staffing situation amongst the clinical workforce, some management participants stated they did not have enough staff or appropriate expertise (often stated as qualitative expertise) in their immediate teams to be able to work effectively to produce meaningful conclusions from the data they received. This was despite an abundance of resource given over to collecting feedback on the ground, leading to a bizarre situation whereby masses of data were being collected from patients, but a lack of skill and personpower, within the patient experience team, prohibited its interpretation and therefore its use: In a drawing together of the points raised so far, it is clear that current patient feedback systems do not generally allow for learning across the organization. The collection of patient experience feedback seems to be the focal point, with an intensive resource given over to this, whilst fractured and disparate teams struggled to make sense of the data or to be able to assist ward staff to do so.

| Atthemicrolevelofthefeedbackitself
Both management and ward staff participants spoke about the usefulness of the patient experience feedback they received. Usefulness was often aligned to whether it was appreciated that improvements could be made based on the feedback. Overall, it was reported that A different problem with the feedback sources currently received related to what extent ward staff were or were not able to interact with and interrogate the raw data which were passed onto them by patient experience team members. Senior ward staff participants were sent spreadsheets of unfiltered and unanalysed feedback. In some instances, this ran into hundreds of rows of text for a month's worth of data. The complexity and volume of the data that ward staff had to contend with were often seen as overwhelming to the extent that some ward staff deliberately chose not to engage with the data. The two main issues which prevented ward staff from using-or some cases even looking-at patient experience feedback were a lack of time and a lack of training. Taking time away from clinical duties to "sift through," a large amount of unsorted data was not perceived to be a high priority. Likewise, it was evident that ward staff did not have the required skills to be able to perform sophisticated analytic tasks on the data they received.
The stark reality is most frontline staff, and even most managers, really struggle to find the time to look at the kind of in-depth reporting we get back. We get reports back that are, you know, extremely bulky documents and people struggle to have the time to really read them, understand them, and use them.
(Trust B, Interviewee 4, Patient experience management) In general, the raw data from patients were said to be difficult for ward staff to interact with and some participants questioned whether the current process was fit for purpose. A few management participants spoke about how a lack of decent analysis before the data were passed onto ward staff simply worked to compound the problem even further. Even more difficult to achieve was the idealized notion that differing data sets should be brought together to provide an overall picture of what patients thought about an individual ward. Despite all of the above difficulties, there was an expectation by senior leaders that ward staff should be using the feedback to make improvements to the ward.
Compounding the above problems of data interrogation, were underlying problems that ward staff perceived to be inherent in the data already collected and therefore its value even before it reached them. Most significantly, timeliness was seen as one of the main concerns with it being difficult to engage ward staff with data that are not real time. A specific example of this is the NHS Inpatient Survey where patient feedback is viewed months after it has been collected.
Frustrations were attached to receiving feedback which was considered historical if ward staff had already started to work on improvements to address known problems. Even FFT data were said to be too late if it reached ward staff a few months after it was collected. Considering the above micro view of the participants' narratives, it can be seen that a large amount of feedback is positive but simultaneously generic in nature. Ward staff struggle to interact with how the feedback is presented to them in its current format, and there are questions raised over the inherent value of the sources, specifically in relation to factors such as timeliness.

| D ISCUSS I ON
From the findings given above, we can see how the ability for effective use to be made of patient experience feedback is hindered A meta principle that can be drawn from the findings of this study is that organizational culture in health care is not changing as fast as actors on the ground strive for it to change. For instance, there is already a recognition that too much data are being collected from patients in relation to the little amount of action that is taken as a result of it. 10,18 Our participants-particularly the management participants-were very mindful of this but largely seemed powerless to prevent the tsunami of ongoing data collection within their organization. Equally, it has been known about for some time that many members of ward staff find interpretation of data sets difficult or impossible as they have minimal or no training in analytics or quality improvement. 18 This issue was raised by both management and ward staff participants in our study, but there was no strategy in place or forthcoming at any of the three organizations we studied to address this issue. The slow movement of culture change discussed above is likely to be related to what has recently been dubbed the "uber-complexity" of health care, 23 with key actors working within a system which favours centralized power structures over localized individualistic solutions.

| Recommendationsforchange
There should be an organizational emphasis placed on the principle that all feedback collected ideally needs to have the ability be meaningfully used by those providing frontline care. Otherwise, it becomes unethical to ask patients to provide feedback which will never be taken into account. An immediate concentration on quality over quantity is important with a strategic focus which takes the priority off the collection of data and onto its use.

| Strengthsandlimitations
To our knowledge, this is the first paper which has paid significant attention to the system-level, macro factors that are inhibiting the use of patient experience feedback. Other commentators [10][11][12]18 have noted some of the micro level factors we have identified here but not how they interact with structural issues problems which further compound the issue at hand. A limitation may be our explicit focus on the problems surrounding the use of patient experience feedback due to the emphasis that participants themselves placed on this aspect. It could be that a write-up-which sought to pay equal attention to problems and solutions-may have uncovered different or more worthwhile suggestions for change.

| CON CLUS ION
Our study found that the use of patient experience feedback is impeded by issues which pertain to both macro level structural/ organizational factors and micro level factors surrounding how individuals interact with the data sources. These factors collide to create a situation where an ever increasing amount and diversity of feedback is being collected, but simultaneously staff at different levels in the hospital hierarchy are struggling to use it to make improvements to patient care. Given the current movement towards the importance of paying attention to patient experience, it is likely that organizational culture and systems are moving too slow in response to how staff say they want to use patient feedback. We put forward recommendations for change which focus on quality over quantity, working towards ensuring ward staff can understand the data they are receiving and changes to organizational structure.

ACK N OWLED G EM ENTS
We would like to thank all the participants who took part in this study.

CO N FLI C TO FI NTE R E S T
The authors declare no conflict of interest.