Keynote Address: United Kingdom Experiences of Evaluating Performance and Quality in Emergency Medicine


  • Suzanne Mason FRCS, FFAEM, MD

    1. From the Director of Health Services Research, School of Health and Related Research, University of Sheffield, Sheffield, UK.
    Search for more papers by this author

  • Funding for this conference was made possible (in part) by 1R13HS020139-01 from the Agency for Healthcare Research and Quality (AHRQ). The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services, nor does mention of trade names, commercial practices, or organizations imply endorsement by the U.S. Government. This issue of Academic Emergency Medicine is funded by the Robert Wood Johnson Foundation.

  • Supervising Editor: James Miner, MD.

  • The authors have no relevant financial information or potential conflicts of interest to disclose.

Address for correspondence and reprints: Suzanne Mason, FRCS, FFAEM, MD; e-mail:


ACADEMIC EMERGENCY MEDICINE 2011; 18:1234–1238 © 2011 by the Society for Academic Emergency Medicine


Demand for emergency care is rising throughout the western world and represents a major public health problem. Increased reliance on professionalized health care by the public means that strategies need to be developed to manage the demand safely and in a way that is achievable and acceptable to both consumers of emergency care, but also to service providers. In the United Kingdom, strategies have previously been aimed at managing demand better and included introducing new emergency services for patients to access, extending the skills within the existing workforce, and more recently, introducing time targets for emergency departments (EDs). This article will review the effect of these strategies on demand for care and discuss the successes and failures with reference to future plans for tackling this increasingly difficult problem in health care.

Crowding in emergency departments (ED) is an international problem for which we have, to date, found no solutions. The majority of interventions that have been tried have not been evidence-based and have been locally implemented rather than aimed at delivering a national or international agenda to encourage change. Change may occur through the modification of help-seeking behavior of patients, the availability of alternative pathways of care for patients, the processes of care once patients are in the ED, or the ease with which patients are able to leave the ED after receiving care. This article will review some of the U.K. evidence and experiences of managing crowding while also trying to deliver high quality patient care in the ED.

Demand for Care

Demand for emergency care is rising and represents a major public health problem. Table 1 indicates the increasing scale of the problem worldwide.1 Rising demand is being driven by public behavior and also changes to health care policy. In the United Kingdom, as in most other westernized countries, the key issues driving increasing demand for emergency care by the public include accessing a higher level of care than is actually required,2,3 partly due to an increased reliance on professionalized health care, and higher public expectations of health care. In addition, increased social mobility and a lack of robust social care structures, along with an aging population, make patients more likely to access emergency care rather than rely on their social networks. Coupled with this, there is an increased ability to deliver efficient and timely emergency care, which may partially drive increasing expectations from the public.

Table 1. 
Mean Annual Rise in ED Attendances by Country
YearsCountryMean Annual Rise (%)
1996–2006United States3.2
2002–2009United Kingdom5.9

In the United Kingdom, health care policy change has also presented challenges in the ability to deliver timely and high-quality emergency care. The government has repeatedly asked for more convenience in health care delivery, with care being taken to the patient, and increased choice in how they access care. This has sometimes had the opposite effect of driving ED attendances up.4 Emergency services, while expanding the role of allied health professionals, have seen junior doctors’ hours of work reduce to a maximum of 48 per week as a result of the European Working Time Directive.5 This creates difficulties in staffing EDs consistently, leading to more challenging working environments and difficulties in delivering high-quality and timely patient care.

Changing Workforce to Manage Demand Better

As mentioned, a number of strategies in the United Kingdom were tried and have failed to manage the rising demand for emergency care. The creation of alternative services that patients could access for advice and management of their urgent health care problem included National Health Service (NHS) Direct (a nurse-led 24-hour telephone helpline) aimed at providing telephone advice for patients, which it was hoped would increase their ability to self-manage minor illness and injury conditions and allow the direction of patients to the most appropriate source of care. Studies have shown that there was no discernable effect of NHS Direct on demand for ED or ambulance service care.4 Similarly, walk-in centers were set up to offset demand for emergency care by giving patients an alternative service they could walk into for emergency care, but no evidence has been found to support this hypothesis.6 In addition, the general practitioners in the United Kingdom renegotiated their contract and were given the option to stop providing 24-hour care for their patients. This created a new gap in services in some parts of the country, where there were insufficient GPs to cover the emergency out-of-hours workload. Patients’ perceptions of their GPs’ availability changed and their default position was to attend the ED.7,8

Other emergency health care services have expanded the remit of health care they provide to try to manage increasing demand better and deliver care closer to the patient. This is especially true of the nation’s ambulance services, which have seen that they can provide a crucial role in this respect, especially for the more vulnerable patients in society, such as the frail elderly. Extended roles for paramedics have evolved whereby patients can be assessed in the home, treated where possible, or sign-posted to alternative sources of care.9,10 However, despite evidence showing the clear benefits of this for patients, the spin-off benefits in reducing ED attendances, and some potential cost savings, these initiatives are only available in patchy geographical areas of the United Kingdom and have not been taken up in a more comprehensive fashion.

Changing Processes of Care

A large study in 200411 tried to identify which organizational factors had deleterious effects on the process of care that patients experienced and were therefore more likely to lead to longer waiting times in U.K. EDs. This very large study used mixed methods, recruited 137 U.K. EDs for the study (representing 65% of U.K. EDs), and found that patients were likely to wait longer in the ED if it had high visit numbers and a caseload skewed toward more serious cases (accounting for 14% variation in mean waiting times). Second, EDs that spent more money on non–pay-related items, such as tests and investigations, had more staff sickness and a less democratic clinical leader and had longer mean waiting times (accounting for an additional 33% of the variation in mean waiting times). Further in-depth qualitative work also demonstrated that waiting times worsened in EDs that lacked boundary spanning behavior, that is, where EDs failed to be proactive in developing working relationships with other agencies on the boundary (such as radiology, laboratories, community care, and the acute ward setting), and thus enhance patient care across the whole journey. In addition, where staff showed higher levels of psychological strain and where there were higher levels of staff autonomy and control, this probably indicates a lack of departmental strategic direction and team working. These factors clearly show that the organization and leadership within an ED can influence factors that affect the level of crowding and patient flow. Further research is needed around interventions to identify the effect of organizational change on crowding and patient care.

Targets in the United Kingdom

One other way of dealing with high levels of demand for services and overcrowded EDs while ensuring efficiency and high quality is to performance manage a service. For many years the U.K. ambulance service has had time-based targets associated with the delivery of care, and this pressure has been brought to bear on EDs in England after several high-profile cases in the media and also with health care policy initiatives by the government. The government mandate for EDs basically stated that “By 2005, 98% of all patients must be in and out of the ED within four hours.”12 The 98% 4-hour cutoff was not based on any good research evidence, and this target had not previously been demanded anywhere else in the world. As a result of the introduction of the target in 2004, overall performance improved as shown in Figure 1. Although the United Kingdom was the first to set a throughput target for ED visits, New Zealand and parts of Australia and Canada are trialing a similar target for their ED patients.13–15

Figure 1.

 England’s 4-hour ED performance, 2002–2008.

To reach this target, EDs undertook a number of measures that were not cost-neutral. These were documented through a survey of English EDs in 2006, to which 111 of 198 EDs responded (56%).16 The commonest measures taken were additional senior doctor hours (39% of EDs respondents), creation of a “4-hour monitor” role (37%), improved access to emergency beds (36%), additional nonclinical staff hours (33%), additional nursing hours (29%), and triage by senior staff (28%). However, in 32% of EDs responding, no changes were made to usual practice. The biggest influence on improved performance during monitoring week was the number of measures that a department took, rather than any specific measure.

The 4-hour target was designed to improve crowding and patient outcomes by reducing the time patients spend in EDs. No one denied that many patients spent longer than needed in our EDs and often because of a lack of an inpatient bed, not because they were receiving essential treatment. However, it was not clear whether putting a cap on time would improve patient outcomes. To explore the effect that the 4-hour target was having on patients, data were analyzed to examine the distribution of total time spent in the ED. Data were available from 83 EDs for the month of April 2004 (n = 428,953 patient episodes). Figure 2 shows the distribution of time in the ED by disposition category of admitted and discharged. The median total time in the ED for discharged patients was 96 minutes (98th percentile = 341 minutes); 91.0% of these patients spent under 220 minutes in the department, with a further 3.6% spending 220 to 239 minutes.

Figure 2.

 Distribution of total time in ED for episodes resulting in admission or discharge, 2004.

Patients admitted from the ED had a median total time of 183 minutes (98th percentile = 625 minutes). The distribution of total time in the department for admitted patients shows the most striking anomaly, with 64.0% of patients spending under 220 minutes in the department and a further 12.3% spending 220 to 239 minutes. Patients spending 220 to 239 minutes in the department were also significantly older than those spending under 220 minutes there.17

Further in-depth work with 15 EDs identified that the 4-hour target did encourage change, and a number of new processes were introduced that were felt to be beneficial to patient care. These included introducing measures such as streaming of patients according to their acuity level on arrival, early ordering of laboratory tests, increased senior doctor triage of patients, and the introduction of clinical decision units where patients could continue their investigations without being admitted to a hospital bed while also “stopping the clock.” Key lessons from the implementation of the target were learned,18 and these included emphasizing the responsibility of the whole organization, maintaining the focus on improving patient care, embracing opportunity, and involving all stakeholders.

A further study analyzed 12.2 million new patient episodes at English EDs from Hospital Episode Statistics data for 2008 and 2009 to see whether the distribution of time in ED had changed. Figure 3 shows that the last 20-minute “spike” in activity was still present and much larger than in 2004, with 30.7% of admitted patients leaving the ED in the 20 minutes before the 4-hour target is breached and 10.5% of discharged patients.

Figure 3.

 Distribution of patient time in ED by admitted and discharged in 2008–2009.

While many in the U.K. specialty of emergency medicine supported the benefits that the 4-hour target produced, it was clear that they were not being experienced by all patients and that processes throughout the hospital and wider health care system may not have improved sufficiently to accommodate it.19 Targets and performance measures in health care are proliferating worldwide as a means to improve the quality and value of care delivered to patients.20–23 However, they do not always hit their mark in improving patient care and can have unintended consequences.24–34 To date, there is no evidence that the 4-hour target benefitted patient care, and indeed the findings would suggest that it has encouraged target-led care rather than needs-led care.

While it is clear that the 4-hour target has resulted in fewer patients spending many hours in the ED, the rising proportion of patients, particularly admissions, who have dispositions in the last 20 minutes of the 4-hour interval strongly suggests that a stringent absolute cutoff may not be the best way to manage ED crowding. The introduction of a stringent cutoff as a target for EDs is a matter of some controversy, and indeed, some institutions have warned against this practice, highlighting that as soon as the target is breached, the incentive is lost, and that crowding near to the cutoff time appears.35

Measuring Quality in Patient Outcomes

The “targets and terror” regimes that have emerged over the past 20 years in the U.K. health service have focused on immediate, easily measurable process interventions to provide quantifiable evidence of performance. This has forced organizations to divert their professional, largely value-driven staff away from “doing the right thing” toward achieving externally imposed goals instead. The Department of Health has recently announced a relaxation of the 98% 4-hour standard, along with the introduction of a dashboard of clinical quality indicators.36 It is likely that additional quality and safety measures will be required to ensure an optimum balance of safety, quality, and timeliness in emergency care.

Measuring outcomes still remains a huge challenge for emergency medicine. This is because outcomes for patients are based on the whole journey, from calling for help to leaving the health care system. This may involve a number of services and health care staff, and to focus attention on one part of that journey, the ED, does not necessarily mean that overall care, and therefore outcomes, is improved. Having reliable data that we can interrogate and manipulate from our EDs is probably the single biggest challenge. Whatever we choose as our outcome or process measures, we need to be able to record and benchmark our activity to measure current activity and inform future improvements. The U.K. experiences with the 4-hour target have led to a better understanding of the challenges that measuring performance can present.

Finally, what outcomes should we measure? We have seen a demonstration of some of the difficulties and dangers associated with having a target or indicator to perform to. The forthcoming challenges for the United Kingdom presented by the new quality indicators presents us with several more opportunities. Change needs to lead to the adoption of a culture that leads to excellent care for all patients, and this should be the case whether our EDs are crowded or not.