This paper was prepared as part of a 2008–2009 objective established for the Society for Academic Emergency Medicine (SAEM) Emergency Department Crowding Task Force by the SAEM Board of Directors. No authors have any financial conflicts of interest with the preparation of the manuscript.
Measures of Crowding in the Emergency Department: A Systematic Review
Article first published online: 13 MAY 2011
© 2011 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 18, Issue 5, pages 527–538, May 2011
How to Cite
Hwang, U., McCarthy, M. L., Aronsky, D., Asplin, B., Crane, P. W., Craven, C. K., Epstein, S. K., Fee, C., Handel, D. A., Pines, J. M., Rathlev, N. K., Schafermeyer, R. W., Zwemer, Jr., F. L. and Bernstein, S. L. (2011), Measures of Crowding in the Emergency Department: A Systematic Review. Academic Emergency Medicine, 18: 527–538. doi: 10.1111/j.1553-2712.2011.01054.x
Supervising Editor: Lawrence M. Lewis, MD.
- Issue published online: 13 MAY 2011
- Article first published online: 13 MAY 2011
- Received August 30, 2010; revisions received November 5 and November 16, 2010; accepted November 18, 2010.
ACADEMIC EMERGENCY MEDICINE 2011; 18:527–538 © 2011 by the Society for Academic Emergency Medicine
Objectives: Despite consensus regarding the conceptual foundation of crowding, and increasing research on factors and outcomes associated with crowding, there is no criterion standard measure of crowding. The objective was to conduct a systematic review of crowding measures and compare them in conceptual foundation and validity.
Methods: This was a systematic, comprehensive review of four medical and health care citation databases to identify studies related to crowding in the emergency department (ED). Publications that “describe the theory, development, implementation, evaluation, or any other aspect of a ‘crowding measurement/definition’ instrument (qualitative or quantitative)” were included. A ”measurement/definition” instrument is anything that assigns a value to the phenomenon of crowding in the ED. Data collected from papers meeting inclusion criteria were: study design, objective, crowding measure, and evidence of validity. All measures were categorized into five measure types (clinician opinion, input factors, throughput factors, output factors, and multidimensional scales). All measures were then indexed to six validation criteria (clinician opinion, ambulance diversion, left without being seen (LWBS), times to care, forecasting or predictions of future crowding, and other).
Results: There were 2,660 papers identified by databases; 46 of these papers met inclusion criteria, were original research studies, and were abstracted by reviewers. A total of 71 unique crowding measures were identified. The least commonly used type of crowding measure was clinician opinion, and the most commonly used were numerical counts (number or percentage) of patients and process times associated with patient care. Many measures had moderate to good correlation with validation criteria.
Conclusions: Time intervals and patient counts are emerging as the most promising tools for measuring flow and nonflow (i.e., crowding), respectively. Standardized definitions of time intervals (flow) and numerical counts (nonflow) will assist with validation of these metrics across multiple sites and clarify which options emerge as the metrics of choice in this “crowded” field of measures.
Crowding is a frequent and pervasive phenomenon for the majority of emergency departments (EDs) in the United States and around the world.1 Crowding occurs when demand for services outstrips available resources. Recent studies have demonstrated that ED crowding is worsening in the US as demonstrated by longer waiting times to see clinicians and is likely exacerbated by the worsening problem of ED boarding, where admitted patients often stay for long periods in the ED waiting for inpatient bed placement.2,3 Crowding adversely affects clinical outcomes, including mortality, and leads to delays in care for time-sensitive conditions, patient dissatisfaction with emergency care, and higher left without being seen (LWBS) rates.4
The most widely accepted conceptual framework of crowding is the input–throughput–output model.5 Input factors are related to the demand for ED services, throughput factors are related to the ED processes of evaluation and treatment, and output factors are related to ED disposition. Despite consensus regarding the conceptual foundation of crowding, and increasing research focused on factors and outcomes associated with crowding, there is no widely accepted way to measure crowding. A review of the medical literature over 6 years ago demonstrated that among the numerous studies published on crowding, a common definition or measure of crowding did not exist.6 Identifying metrics that are feasible, accurate, and reproducible may enable clinicians, administrators, researchers, and policy makers to better understand and manage ED crowding. The purpose of this study was to conduct a systematic review of all existing crowding measures and compare them in terms of their conceptual foundations and validity.
We conducted a systematic, comprehensive review of four medical and health care citation databases to identify all studies related to crowding in the ED. Three independent reviewers screened the titles and abstracts from the comprehensive search and selected all studies that focused on the development or validation of a measure(s) of crowding in the ED. A second set of reviewers scanned the full-text versions of the papers to verify that the study proposed and/or examined the validity of a crowding measure. Finally, a third set of reviewers examined all of the crowding measures in the eligible papers and summarized the evidence of their validity.
In collaboration with a medical librarian (CC) from the Welch Medical Library at the Johns Hopkins University School of Medicine, we developed and executed a systematic search strategy that allowed us to conduct a comprehensive literature review by separately searching PubMed (MEDLINE), CINAHL, Embase, and the Cochrane Database of Systematic Reviews (Cochrane Collaboration) to identify all scientific articles that were published or available on-line between January 1, 1966, and September 22, 2009, and related to the concepts of “emergency department” AND “crowding.” The search strategy we used for PubMed, CINAHL, and Embase included creating database-specific queries that included official controlled vocabulary terms for each relevant concept when available, as well as keywords and keyword phrases for each concept (see Table 1). Controlled vocabulary terms were incorporated into the search queries for three of the databases: Medical Subject Headings [MeSH] for MEDLINE, CINHAL Headings for CINHAL, and Emtree terms for Embase. For the Cochrane Database of Systematic Reviews, only keywords and keyword phrases were used in the query; they are sufficient for comprehensive retrieval within this resource given its limited total number of records (see Table 1).
|ED Concept||Crowding Concept|
|Controlled Vocabulary Term||Key Words||Controlled Vocabulary Term||Key Words|
|PubMed||Emergency Service, Hospital [MeSH] OR Emergency Medical Services [MeSH] OR Emergency Medicine [MeSH]||“emergency department” OR “emergency room” OR “ER”||Crowding [MeSH] OR Surge Capacity [MeSH]||crowd* OR overcrowd *”surge capacity” OR surge OR surges OR diversion OR occupancy OR congestion|
|CINAHL||Emergency Service+ OR Emergency Medical Services+ OR Emergency Medicine||“emergency department” OR “emergency room” OR “ER”||Crowding||crowd* OR “surge capacity” OR surge OR surges OR overcrowd* OR diversion OR occupancy OR congestion|
|EMBASE||emergency ward/exp OR emergency health service/exp OR emergency medicine/exp||“emergency ward” OR “emergency health service” OR “emergency medicine” OR “emergency department” OR “emergency room” OR ER OR “emergency medical services” OR “emergency service, hospital”||crowding/exp OR hospital bed utilization/exp||crowding OR “surge capacity” OR crowd* OR overcrowd OR surge OR surges OR diversion OR “hospital bed utilization” OR “bed utilization” OR occupancy OR congestion OR “bed utilisation”|
|Cochrane Database of Systematic Reviews||N/A||“emergency department” OR “emergency room” OR “emergency services”||N/A||crowd OR crowding OR overcrowding OR crowds OR diversion|
The titles and abstracts of all papers related to crowding in the ED were reviewed to determine whether the study included a measure of crowding. Unless otherwise specified, scale, measure, and definition were regarded as synonymous and are referred to simply as “measure” throughout this paper. Although the search strategy included the key word “surge” or key word phrases that included “surge,” the studies were only considered eligible if “surge” described daily operational surge; papers related to disaster or mass casualty surge were excluded. Papers not published in English were also excluded.
Three independent reviewers (PC, SE, UH) completed an initial screen of all titles and available abstracts from the comprehensive search using the following inclusion criteria: papers with titles or abstracts that describe the theory, development, implementation, evaluation, or any other aspect of a “crowding measurement/definition” instrument (qualitative or quantitative) and a “measurement/definition” instrument is anything that involves the act or process of assigning a value to the phenomenon of crowding in the ED.
Following the same inclusion criteria, a second set of reviewers (MM, NR, RS) independently evaluated the full text versions of the papers and judged whether each study was appropriate for inclusion. Studies were included if at least two out of the three reviewers deemed them eligible. The reference sections of these full-text papers were also examined to ensure publications were not missed during the comprehensive literature search of the four databases.
The final set of papers, having majority agreement for inclusion from the first and second screeners, were reviewed for information on ED crowding measures. A data abstraction instrument was developed (DA, CF, DH, UH, MM, JP, RS) to record information about the final set of papers. Out of a pool of six investigators (DA, CF, DH, JP, NR, FZ), three randomly assigned reviewers abstracted descriptive information from each publication. Publications that involved an identified or self-declared reviewer’s conflict of interest were reassigned. The abstraction instrument included categories concerning the study design, objective, crowding measure, and evidence of validity.
Based on data abstracted by the independent reviewers, two additional reviewers (UH and MM) reviewed all of the articles abstracted. Through conference calls and threaded e-mail discussions, these two reviewers discussed areas of disagreement, and consensus was reached on categorization of all the measures into one of five measure types (clinician opinion, input factors, throughput factors, output factors, or multidimensional scales), and indexed these to six comparative validation criteria (clinician opinion-consensus panel, ambulance diversion, LWBS rate, times to care, forecasting [predicting future levels of crowding], and other [i.e., mortality, risk of methicillin-resistant Staphylococcus aureus infection, or opportunity loss of treatment capacity]).
A total of 2,660 papers were identified using the described databases and terms, of which 747 were unique papers that focused on measuring crowding in the title and abstract. After the initial screening, 92 titles and abstracts met inclusion criteria for addressing the development or validation of measures of crowding in the ED. After the second screening of the full-text version of these publications, a total of 70 met inclusion criteria for data abstraction (i.e., after review of the full-text version of the publication, 22 papers were found to not address the development or validation of measures of ED crowding). Of the 70 publications that were reviewed, 46 were original research papers (see Figure 1). Upon review of the full-text versions of the papers, 24 of the 70 were excluded for the following reasons: five were concept papers,5,7–10 five were editorials,11–15 three were reports,16–18 six were review papers,19–24 one was a website for an ED crowding measure calculator,25 one was a performance standard,26 one was an abstract,27 and two were not available in English.28,29
The 46 original studies contained 71 unique crowding measures. These measures were categorized into the five types of measures and indexed to their respective comparative validation criteria. Of the types of measures, there are three clinician opinion measures, 17 input measures, 21 throughput measures, 21 output measures, and nine multidimensional measures. A summary of the results is provided in Table 2.31–74
|Measure Type||Measure||Validation Criteria|
|Clinician Opinion, Consensus Panel (Reference Number[s])||Ambulance Diversion (Reference Number[s])||LWBS (Reference Number[s])||Times to Care (Reference Number[s])||Forecasting (Reference Number[s])||Other (Reference Number[s])||Detailed Comments|
|Clinician opinion||Physicians feel rushed31,32||31, 32||Significantly associated with clinician opinion of crowding.31,32|
|Clinician opinion of crowding30||30||Significantly associated with LWBS.30|
|EP satisfaction47||47||Significantly associated with clinician opinion of crowding.47|
|Input||Waiting time31–36||31, 32||35||36||33, 34||Not significantly associated with diversion.35 Two-hour-ahead forecasts good but relatively poor by 8 hours.33,34|
|Waiting room filled > 6 hours/day31,32||31, 32||Significantly associated with clinician opinion of crowding.31,32|
|Time to physician37,47,57||47||57||37||Significantly associated with clinician opinion of crowding,47 and LWBS (did not wait).57 No validation, measured trends over time.37|
|No. of arrivals38–41||41||38||39||40||Not associated with diversion.41 Significantly associated with LWBS,38 wait time, boarding time, and ED LOS.39 Lead indicator of ED census and diagnostic resources.40|
|No. of pts in waiting room31–36,42–44||31, 32, 42||35||36, 43, 44||33, 34||Significantly associated with clinician opinion of crowding,31,32,42 diversion,35 waiting room time,36 and ED LOS.44 Two- and 8-hour-ahead forecasts good to moderate, respectively.33,34|
|No. of pts registered35,42||42||35||Significantly associated with clinician opinion of crowding,42 but not diversion.35|
|No. or % of ambulance pts registered45||45||Significantly associated with clinician opinion of crowding.45|
|No. of pts awaiting triage42||42||Significantly associated with clinician opinion of crowding.42|
|No. of low-complexity pts46||46||Negligible increase in time to physician or ED LOS.46|
|No. of pts at each acuity level37||37||No validation, measured trends over time.37|
|Average triage acuity level45||45||Significantly associated with clinician opinion of crowding.45|
|No. of new pts by usual care45||45||Significantly associated with clinician opinion of crowding.45|
|Percentage of open appointments in ambulatory care clinics45||45||Significantly associated with clinician opinion of crowding.45|
|LWBS35,37||35||37||Not associated with diversion.35 No validation, measured trends over time.37|
|Average or % of pts who leave without treatment complete45||45||Significantly associated with clinician opinion of crowding.45|
|Ambulance diversion episodes45,69||45, 69||Significantly associated with clinician opinion of crowding.45,69|
|Average EMS waiting time45||45||Significantly associated with clinician opinion of crowding.45|
|Throughput||ED beds at capacity > 6 hours or hallways filled > 6 hours31,32||31, 32||Significantly associated with clinician opinion of crowding.31,32|
|Percentage of time ED ≥ stated capacity47||47||Significantly associated with clinician opinion of crowding.47|
|No. of full rooms42||42||Significantly associated with clinician opinion of crowding.42|
|Total no. of pts in ED37,40,42,43,47–52||42, 47, 48||43||43||49, 50||40, 51, 52||37||Significantly associated with clinician opinion of crowding.42,47,48 No validation, measured trends over time.37 Lead indicator of ED census and diagnostic resources.40 Not associated with daily mean ED LOS,50 nor daily median ED LOS.49 Not associated with diversion or LWBS.43 ED census is cyclical.52|
|ED occupancy rate33,34,45,47,53–55||45, 47||53, 54||53||33, 34, 54, 55||Significantly associated with clinician opinion of crowding.45,47 Good discriminator of diversion53,54 and LWBS.53 Good forecasts 2, 8, and 12 hours ahead.33,34,55|
|No. of hallway pts42||42||Significantly associated with clinician opinion of crowding.42|
|No. of resuscitations in past 4 hours43||43||43||Not associated with diversion or LWBS.43|
|No. of pts being treated36,44||36, 44||Significantly associated with waiting room time36,44 and treatment time.44|
|No. of pts waiting for specialty consult or disposition by consultant > 4 hours43||43||43||43||Significantly associated with boarding time but not diversion or LWBS.43|
|No. of ED diagnostic orders40||40||Short-term forecasts poor but a significant predictor of future ED census.40|
|No. of pts waiting test results36||36||Significantly associated with waiting room time.36|
|No. of nurses working43||43||43||No relationship noted with diversion or LWBS.43|
|Pts treated by acuity per bed hours45||45||Significantly associated with clinician opinion of crowding.45|
|No. of pts per nurse or physician45,48||45, 48||Significantly associated with clinician opinion of crowding.45,48|
|No. of pts admitted or discharged per physician45||45||Significantly associated with clinician opinion of crowding.45|
|Sum of pt care time per shift56||56||Significantly associated with mortality within 10 days of ED visit.56|
|ED ancillary service turnaround time42,45||42, 45||Only one of the two studies associated with clinician opinion of crowding.45|
|Time to consultation37||37||No validation, measured trends over time.37|
|Time to room placement37,45||45||37||Significantly associated with clinician opinion of crowding.45 No validation, measured trends over time.37|
|ED treatment time41||41||Not associated with diversion.41|
|ED LOS33–35,45,47||45, 47||35||33, 34||Significantly associated with clinician opinion of crowding.45,47 Not associated with diversion.35 Good forecasts 2 and 8 hours ahead.33,34|
|Output||No. or % of admissions38,39,49,50,70||38||39, 49, 50||70||Significantly associated with LWBS,38 waiting room time,39 and ED LOS.39,49,50 Poor short-term forecasts.70|
|No., mean no., or % of boarders33–36,38,39,41,42,44,45,47,57||42, 45, 47||35, 41||38||36, 39, 44||33, 34||57||Significantly associated with clinician opinion of crowding,42,45,47 diversion,35,41 LWBS,38 waiting room time,36,44 treatment time,39,44 boarding time,39,44 and ED LOS.39 Not associated with hospital mortality or MRSA infections.57 Good forecasts at 2 and 8 hours ahead.33,34|
|Boarding time33,34,42,43,45,47,57–59||42, 45, 47, 58||43||43||33, 34||57, 59||Significantly associated with clinician opinion of crowding,42,45,47,58 diversion,43 LWBS,43 and opportunity loss of treatment capacity.59 Not associated with hospital mortality or MRSA infections.57 Short-term forecasts good but underestimated boarding time.33,34|
|Boarding time components45||45||Significantly associated with clinician opinion of crowding.45|
|Observation unit census45||45||Significantly associated with clinician opinion of crowding.45|
|No. of pts waiting discharge ambulance pick-up36||36||Significantly associated with no. of pts waiting to be seen.36|
|ED admission transfer rate45||45||Significantly associated with clinician opinion of crowding.45|
|Hospital admission source45||45||Significantly associated with clinician opinion of crowding.45|
|Inpatient occupancy level38–40,45,49||45||38||39, 49||40||Significantly associated with clinician opinion of crowding,45 LWBS,38 treatment time,39 boarding time,39 and ED LOS.39,49 Not a significant predictor of future ED census.40|
|Hospital supply/demand forecast 45||45||Significantly associated with clinician opinion of crowding.45|
|ED volume/inpatient bed capacity45||45||Significantly associated with clinician opinion of crowding.45|
|No. of inpatients ready for discharge45||45||Significantly associated with clinician opinion of crowding.45|
|Number of staffed acute care beds47||47||Significantly associated with clinician opinion of crowding.47|
|Inpatient processing times45||45||Significantly associated with clinician opinion of crowding.45|
|Inpatient laboratory, radiology, CT orders40||40||Not a significant predictor of future ED census.40|
|Time from request to bed assignment47||47||Significantly associated with clinician opinion of crowding.47|
|Time from bed ready to ward transfer47||47||Significantly associated with clinician opinion of crowding.47|
|Agency nursing expenditures45||45||Significantly associated with clinician opinion of crowding.45|
|Local home care service availability69||69||Significantly associated with clinician opinion of crowding.69|
|Alternate level of care bed availability69||69||Significantly associated with clinician opinion of crowding. 69|
|Nearby EDs diverting ambulances69||69||Significantly associated with clinician opinion of crowding. 69|
|Multidimensional indices||EDWIN53,54,60,66–68||66–68||53, 54, 60||53||Significantly associated with clinician opinion of crowding,66–68 diversion,53,54,60,66 and LWBS.53|
|NEDOCS54,60–65||61, 63–65||54, 60||62||Significantly associated with clinician opinion of crowding61,64,65 (except for one study63), diversion,54,60 and LWBS.62|
|Pediatric NEDOCS (PEDOCS)64||64||Significantly associated with clinician opinion of crowding but outperformed by two operational variables.64|
|READI54,65,67||65, 67||54||Moderate67 to poor association with clinician opinion of crowding65 and not a good discriminator of diversion.54|
|EDCS67||67||Not a good discriminator of clinician opinion of crowding.67|
|ED Work Score54,71||54, 71||Significantly associated with ambulance diversion.54,71|
|Critical Bed Status (CBS)72||72||Significantly associated with time to room but not other ED process of care times.72|
|System complexity73||Not validated against any criteria.73|
|Overcrowding Hazard Scale74||74||Significantly associated with mortality ≤2, 7, and 30 days of ED visit.74|
Prevalence of Measures
Clinician opinion, or perception of ED crowding, was the least commonly used type of crowding measure. Four papers included three types of clinician opinion measures; only one evaluated this type of measure against an objective outcome (LWBS rate),30 and the remaining were based on ED director survey or consensus opinion.
Input measures ranged from waiting times, to number or percentage of patients as they arrived to the ED, to patient severity and complexity (e.g. number of patients at each acuity level). Of these, the most commonly described measures were numerical counts or percentage of patients (as arrivals, in the waiting room, at triage or registration, by acuity, etc.), which were studied against all six types of validation criteria in 16 different papers.31–46
Throughput measures included from ED capacity measures, numerical counts, or percentages of patients in the ED at various stages of ED evaluation, patient care times, and ED length of stay (LOS). Of these, the most commonly used measures were total number of patients in the ED (in 12 papers),36,37,40,42–44,47–52 ED occupancy rate (in seven papers),33,34,45,47,53–55 and times associated with patient care (nine papers).33–35,37,41,42,45,47,56
Output measures included hospital measures of numerical counts, mean values or percentages of admissions, patients boarding in the ED, hospital beds and census, and times of care to leave the ED. Of these, the most commonly used measures were number, percentage, or mean number of boarders (in 13 papers) and boarding times (in nine papers).33,34,42,43,45,47,57–59
Finally, of the multidimensional indices, the most frequently studied measures were the National ED Overcrowding Study (NEDOCS) scale (in seven papers)54,60–65 and the Emergency Department Work Index (EDWIN) measure (in six papers);53,54,60,66–68 both were validated against each other and outcomes of clinician opinion, ambulance diversion, and LWBS rates.
Validation of Measures
Only one clinician opinion measure was validated against objective criteria and showed a positive relationship with LWBS rates.30 The three most commonly proposed input measures were the total number of patients in the waiting room, waiting room time, and the total number of arrivals. Both the total number of patients in the waiting room and the total number of ED arrivals were positively correlated with ED process times such as waiting room time and ED LOS. Furthermore, one study found the number of ED arrivals was an important leading indicator of future ED census and demand for diagnostic resources.40
The most commonly proposed throughput measures of crowding were ED census (total number of patients in the ED), ED occupancy rate, and ED LOS. All three measures were correlated with clinician opinion of crowding. The ED occupancy rate was also positively associated with ambulance diversion and LWBS rates. When measured daily, however, one study did not find that daily ED census was a significant predictor of daily mean ED LOS.50
The number or percentage of ED admissions; the number, mean number, or percentage of boarders; boarding time; and inpatient occupancy levels were the most common output measures proposed. ED admissions, boarders, and inpatient occupancy levels were significantly correlated with ED process times in addition to clinician opinion of crowding, ambulance diversion, and LWBS. Short-term forecasts of the number of boarders (i.e., 2 and 8 hours ahead) were more reliable than short-term forecasts of boarding time.33,34
Of the multidimensional measures, EDWIN66 and NEDOCS61 have demonstrated the most evidence of validity in terms of their positive association with clinician opinion of crowding, ambulance diversion, and LWBS. However, in a 1-year study period at six EDs, investigators found that the ED occupancy rate discriminated as well as EDWIN during hours when EDs were on ambulance diversion or had one or more patients LWBS.53 Similarly, investigators found that the ED occupancy rate discriminated ambulance diversion episodes as well as other multidimensional measures at their facility during an 8-week study period.54
Considerable attention has been devoted to the phenomenon of crowding in EDs over the past decade. Numerous measures have been proposed and developed, and there is growing consensus of the need for quantitative, objective crowding measures that can be used across multiple sites and that are feasible and reproducible. The objective of this study was to conduct a systematic review of all existing crowding measures and compare them in terms of their conceptual foundation and validity. Seventy-one unique measures of ED crowding were identified in the medical literature. The vast number and wide variability in the metrics of crowding reflect how challenging this common phenomenon is to measure.
The results of this review suggest that time intervals and numerical counts are becoming the most prominent measures of crowding in the medical literature. Furthermore, these categories of metrics represent the divergence of crowding measurement into two separate but related phenomena: patient flow and nonflow. The patient flow category relies predominantly on time intervals (e.g., ED total LOS and boarding time). Although the patient flow metrics are more challenging to observe in real time, they appear to be more generalizable across sites. The second category of metrics, nonflow, addresses the traditional concept of ED crowding, predominantly through the use of numerical counts of patients (e.g., ED census, number of waiting room patients, and number of boarders). Based on this review, we believe that it is most intuitive to think of these metrics as measures of nonflow. The advantage of the numerical patient counts is that they are easier to observe in real time; they may be, however, more challenging to generalize across multiple EDs.
The use of simpler measures of crowding appears to have come full circle. Early studies included surveys of ED providers and simple measures of census and ED boarding. The next series of articles focused on the development of multidimensional scales using real-time census, staffing, patient acuity, and hospital variables. Unfortunately, the challenge of capturing ED-specific variation across and within multiple EDs as they transition between normal and crowded conditions was extremely difficult. This challenge may also be difficult to overcome with increasing sophistication of the scales. The common theme of the multidimensional scales includes the observation that they work well in the EDs where they were derived. There is need for evidence of their scalability across multiple EDs. The inability of all EDs to capture the many components of these complex scales, however, may be a disadvantage.
From this review, we do not believe that complex ED work scales, even the extensively studied EDWIN66 and NEDOCS,61 will generalize well enough across diverse settings for them to serve as criterion standard measures of ED crowding. We recommend that attempts to develop more complex multidimensional workload scales should instead turn and focus on further validation and investigation of less complicated measures. The transition back to simple and objective measures of crowding may be more practical. These measures are easier to collect and conceptually are more reproducible. While many of the more straightforward measures have greater reproducibility as objective metrics (e.g., number of patients, ED LOS), others may be less so because of their subjective nature and being site-specific (e.g., physicians feeling rushed, critical bed status). We suggest that future efforts to study and develop crowding measures be directed at using time intervals (flow) and counts (nonflow).
Numerical counts (nonflow) such as total ED census, waiting room patients, and the number of boarders will be useful measures, particularly for point-in-time “snapshots” of conditions within a single ED. Real-time counts are better used to test hypotheses prospectively and are more amenable for gauging conditions over time within a single institution when new processes are implemented. It is unclear, however, how easily numerical counts will generalize across sites effectively. In comparisons across sites, numerical counts will need to be expressed as a percentage (or quantiles) of a predicted count for each ED or as rates such as occupancy (census divided by number of standardized ED beds). The relationship between time intervals (flow) and how these gauge processes within a system is conceptually different from what is measured by numerical counts (nonflow). Measures of flow may be more generalizable across sites and thus have increased utility for multisite comparisons. Flow measures, however, are typically retrospective in nature and may be better used to calculate associations and generate hypotheses. Many organizations engaged in performance improvement projects rely on timed measures of patient throughput and key operational turnaround times.75–77 While time intervals are not considered a crowding metric by these agencies,75–77 the ultimate goal remains measurement of “flow.”
Measures of flow and nonflow (crowding) are not mutually exclusive. Both concepts are worth measuring, not only in single sites, but also as comparisons across sites. Numerical counts (as a percentage of allocated resources) and process times are likely to be linked both as predictors of crowding and as outcomes of crowding. It is very likely they measure different aspects of the phenomenon of ED crowding. Use of both flow and nonflow metrics highlight the fact that the phenomenon of crowding is not shouldered by the ED alone, but is also dependent on hospital- or systemwide factors. Time interval performance measures of factors outside the ED such as diagnostic efficiency (e.g., laboratory and radiology turnaround times), consult times, operating room activity, and inpatient bed availability, help to complement factors within the ED, such as patient counts, to provide a more complete picture of both the causes and effects of crowding. Ultimately the usefulness of these measures will be determined by the extent to which they inform the system that is being investigated and how well they translate across settings. Consequences of crowding relevant to patients, clinicians, researchers, administrators, and policy makers include clinical outcomes, patient safety, patient and staff satisfaction, and cost of care.
Another important finding of this review was the diversity of metrics that were conceptually measuring the same thing. For example, ED census was also referred to as total number of patients registered or total number of patients in the ED. Some studies measured total number of arrivals,38–41 while others used number of patients in the waiting room,31–36,42–44 the number of patients registered,35,42 or the number of patients at triage.42 Other studies used ED patient process times37,41,42,45 or LOS.33–35,45,47 On a practical level, each of these metrics measures something different. Theoretically, however, they are all input and throughput measures using the same unit: numbers or time intervals. We believe that a standardized approach, perhaps using more simple methods of both time intervals (flow) and patient counts (nonflow), would be extremely helpful. Standardization of measures would give clinicians, researchers, administrators, and policy makers the ability to compare and contrast crowding using similar references and units. This would also support future studies with the measurement and interpretation of crowding both within and across multiple EDs.
The heterogeneous nature of the ED crowding literature and studies of the factors causing and resulting from it (surrogate measures) may have resulted in misclassification of papers, study objectives, and measures. Systematically reviewing and summarizing proposed crowding measures was difficult, even for a group of emergency physicians and researchers with significant experience in studying and managing crowding. There were often disparities in the interpretation of results and measures, as some references included for final review did not focus on the goal of developing an instrument to measure crowding, but instead evaluated crowding measures in terms of outcomes (e.g., LWBS rate, ED LOS, mortality), were pre–post-intervention studies designed to alleviated crowding in EDs, or were descriptive surveys about crowding itself. This review was limited to evaluating the conceptual foundations and evidence of validity of the different measures. It did not evaluate the measures in terms of their reliability or responsiveness. This was largely because of the paucity of data available on these two traits for the majority of measures.
While there remains no objective criterion standard measure of crowding in the ED, a combination of time intervals and patient counts appears to be emerging as the most promising tools for measures of flow and non-flow (i.e., crowding), respectively. Crowding scales that use multiple flow and nonflow variables simultaneously have been developed, but their validity has not been reproducible outside of the settings where they were derived. Attempts to create additional complex multidimensional scales are unlikely to overcome this issue. Standardized definitions of time intervals (flow) and numerical counts (nonflow) will assist with validation of these metrics across multiple sites. Ultimately the usefulness of measures will be determined by the extent to which they inform priority outcomes for the system, such as clinical outcomes, patient safety, patient and staff satisfaction, and costs of care. The validity and feasibility of both flow and nonflow metrics will clarify which options emerge as the metrics of choice in this “crowded” field of measures.
- 1American College of Emergency Physicians. Policy statements: crowding. Ann Emerg Med. 2006; 47:585.
- 9A priority queuing model to reduce waiting times in emergency care. Int J Health Care Qual Assur. 1996; 9:10–6., , .
- 16Australasian College of Emergency Medicine. Access Block and Overcrowding in Emergency Departments. Available at: http://www.acem.org.au/media/Access_Block1.pdf. Accessed Feb 26, 2011.
- 17United States General Accounting Office. Hospital Emergency Departments: Crowded Conditions Vary Among Hospitals and Communities. Available at http://www.gao.gov/cgi-bin/getrpt?GAO-03-460. Accessed Feb 26, 2011.
- 20Agency for Healthcare Rsearch and Quality. Researchers identify measures of workflow that may help to explain and mange emergency department overcrowding. AHRQ Res Activities 2004; 282:10–1.
- 24Analysis of the literature on emergency department throughput. West J Emerg Med. 2009; 10:104–9..
- 25NEDOCS Calculator. Available at: http://www.nedocs.org/NEDOCS_Calculator.aspx. Accessed Apr 12, 2011.
- 26The Joint Commission. JCAHO standard LD.3.11. In. Available at: http://www.jointcommission.org/standards_information/standards.aspx. [Requires Access Code.] Accessed Feb 26, 2011.
- 27Visualizing temporal patterns of demand, throughput, and crowding in an emergency department. AMIA Annu Symp Proc. 2007:888., .
- 28The real-time bed occupancy rate: relevance to emergency unit evaluation. Reanim Urgences. 1997; 6:621–7., , , , , .
- 29Emergency department overcrowding: quantification of associated factors. Med Clin. 2003; 121:167–72., , , et al.
- 41Root Cause Analysis of Emergency Department Crowding and Ambulance Diversion in Massachusetts. A report submitted by the Boston University Program for the Management of Variability in Health Care Delivery. Available at: http://www.mass.gov/Eeohhs2/docs/dph/quality/healthcare/ad_emergency_dept_analysis.pdf. Accessed Feb 26, 2011., , .
- 42Development of a site sampling form for evaluation of ED overcrowding. Med Sci Monit. 2002; 8:CR549–53., , , , , .
- 60An early warning system for overcrowding in the emergency department. AMIA Ann Symp Proceed. 2006:339–43., .
- 69Urban emergency department overcrowding: defining the problem and eliminating misconceptions. CJEM. 2002; 4:76–83., , .
- 75Institute for Healthcare Improvement. Patient Flow: Measures. Available at: http://www.ihi.org/IHI/Topics/Flow/PatientFlow/Measures/. Accessed Mar 4, 2011.
- 76National Quality Forum. NQF endorses measures to address care coordination and efficiency in hospital emergency departments. Available at: http://urgentmatters.org/media/file/NQF%20Press%20Release.pdf. Accessed April 12, 2011.
- 77Institute of Medicine. Hospital-based Emergency Care: At the Breaking Point. Washington, DC: The National Academies Press, 2006.