This work was supported in part by grants 1K01HS017957-02 and 1R13 HS 20139-01 from the Agency for Healthcare Research and Quality. The views expressed in this publication do not necessarily reflect the official policies of the Department of Health and Human Services, nor does mention of trade names, commercial practices, or organizations imply endorsement by the U.S. Government. This issue of Academic Emergency Medicine is funded by the Robert Wood Johnson Foundation.
Comparison of Methods for Measuring Crowding and Its Effects on Length of Stay in the Emergency Department
Article first published online: 13 DEC 2011
© 2011 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Special Issue: Proceedings of the 2011 AEM Consenus Conference: Interventions to Assure Quality in the Crowded Emergency Department Guest Editors: James R. Miner, MD Manish N. Shah, MD, MPH
Volume 18, Issue 12, pages 1269–1277, December 2011
How to Cite
McCarthy, M. L., Ding, R., Pines, J. M. and Zeger, S. L. (2011), Comparison of Methods for Measuring Crowding and Its Effects on Length of Stay in the Emergency Department. Academic Emergency Medicine, 18: 1269–1277. doi: 10.1111/j.1553-2712.2011.01232.x
The authors have no potential conflicts of interest to disclose.
Supervising Editor: James Miner, MD.
- Issue published online: 13 DEC 2011
- Article first published online: 13 DEC 2011
- Received August 11, 2011; accepted August 12, 2011.
ACADEMIC EMERGENCY MEDICINE 2011; 18:1269–1277 © 2011 by the Society for Academic Emergency Medicine
Objectives: This consensus conference presentation article focuses on methods of measuring crowding. The authors compare daily versus hourly measures, static versus dynamic measures, and the use of linear or logistic regression models versus survival analysis models to estimate the effect of crowding on an outcome.
Methods: Emergency department (ED) visit data were used to measure crowding and completion of waiting room time, treatment time, and boarding time for all patients treated and released or admitted to a single ED during 2010 (excluding patients who left without being seen). Crowding was characterized according to total ED census. First, total ED census on a daily and hourly basis throughout the 1-year study period was measured, and the ratios of daily and hourly census to the ED’s median daily and hourly census were computed. Second, the person-based ED visit data set was transposed to person-period data. Multiple records per patient were created, whereby each record represented a consecutive 15-minute interval during each patient’s ED length of stay (LOS). The variation in crowding measured statically (i.e., crowding at arrival or mean crowding throughout the shift in which the patient arrived) or dynamically (every 15 minutes throughout each patient’s ED LOS) were compared. Within each phase of care, the authors divided each individual crowding value by the median crowding value of all 15-minute intervals to create a time-varying ED census ratio. For the two static measures, the ratio between each patient’s ED census at arrival and the overall median ED census at arrival was computed, as well as the ratio between the mean shift ED census (based on the shift in which the patient arrived) and the study ED’s overall mean shift ED census. Finally, the effect of crowding on the probability of completing different phases of emergency care was compared when estimated using a log-linear regression model versus a discrete time survival analysis model.
Results: During the 1-year study period, for 9% of the hours, total ED census was at least 50% greater than the median hourly census (median, 36). In contrast, on none of the days was total ED census at least 50% greater than the median daily census (median, 161). ED census at arrival and time-varying ED census yielded greater variation in crowding exposure compared to mean shift census for all three phases of emergency care. When estimating the effect of crowding on the completion of care, the discrete time survival analysis model fit the observed data better than the log-linear regression models. The discrete time survival analysis model also determined that the effect of crowding on care completion varied during patients’ ED LOS.
Conclusions: Crowding measured at the daily level will mask much of the variation in crowding that occurs within a 24-hour period. ED census at arrival demonstrated similar variation in crowding exposure as time-varying ED census. Discrete time survival analysis is a more appropriate approach for estimating the effect of crowding on an outcome.
Status of Measuring Crowding in the ED
During the past decade, advancements have been made in the measurement of crowding in the emergency department (ED). Historically, crowding was most often measured by the frequency of ambulance diversion or by provider opinion surveys.1–3 However, these measures were not highly reliable because EDs use different criteria to trigger ambulance diversion, and provider opinion questions were not standardized or well validated. With the uptake of ED information systems (EDIS), and the appreciation that crowding needs to be measured in a more objective and reliable way, numerous quantitative measures have been proposed.4 The majority of these recent measures characterize crowding in terms of an input (e.g., number of arrivals), throughput (e.g., number of patients being treated), or output (e.g., boarding time) factor.4,5 Some crowding measures are multidimensional, such as the Emergency Department Work Index (EDWIN) or the National Emergency Department Overcrowding Scale (NEDOCS).6,7 Rarely have the different crowding measures been directly compared. The few comparisons performed suggest that the measures may be interchangeable; they yield similar conclusions regarding the effect of crowding on different outcomes.8–11 However, one meaningful difference is that some measures are more feasible for routine use because they require fewer data elements, and the data elements required are commonly captured electronically even in facilities that do not have a comprehensive EDIS.8
Despite the number of crowding measures developed in recent years, little attention has been paid to the appropriate methods of measuring crowding and quantifying the effects of crowding on ED length of stay (LOS). Should crowding be measured on a daily or hourly basis? Should crowding be measured statically, such as using crowding at the time of a patient’s arrival, or pooled over a period of time? Or should it be measured dynamically, acknowledging that the degree of crowding varies throughout a patient’s ED stay? Does the effect of crowding on an outcome change over time? This presentation from the June 2011 Academic Emergency Medicine consensus conference “Interventions to Assure Quality in the Emergency Department” compares methods of measuring crowding and provides insight into the strengths and limitations of different approaches.
Characteristics of the Study Site (Table 1)
|Annual ED volume||58,517|
|Annual median admission rate||22.5%|
|Annual median left without being seen rate||6.0%|
|ED treatment capacity||39|
|Main ED treatment spaces||31|
|Fast track treatment spaces||8 (open daily between 8AM-2AM)|
|Median service completion times, minutes (IQR)|
|Waiting room time||24 (11–64)|
|Treatment time||203 (120–341)|
|Boarding time||108 (52–205)|
|Crowding measured by total ED census|
|Low crowding (0–24th percentile)||6–27|
|Medium crowding (25th–74th percentile)||28–46|
|High crowding (≥75th percentile)||47–90|
|Median staffing per hour (IQR)|
|Fellows/residents/physician extenders||4 (2–5)|
To compare crowding methods, we used 2010 ED visit data from a community teaching hospital affiliated with a major academic center. The hospital has an acute care inpatient bed capacity of 318 licensed and staffed beds. The study ED consists of 31 treatment spaces in the main ED and an additional eight spaces in an adjacent urgent care area. There is a remote observation unit that is staffed by inpatient physicians. Both adult and pediatric patients are treated at the study ED.
The ED volume in 2010 was 58,517 and the mean (±SD) LOS was 6.3 (±5.4) hours for discharged patients and 8.9 (±5.5) hours for admitted patients. The study ED is primarily staffed by physicians and physician assistants. Residents from the affiliated academic teaching hospital work in the study ED, but are not there 24/7. The ED uses MEDITECH Client Server Version 5.54 (Westwood, MA) for all patient care and tracking activities except physician documentation. The primary providers document using paper template charts, which are scanned and stored electronically in the EDIS. MEDITECH interfaces with all of the hospital’s other electronic clinical information systems and is also the inpatient information system.
From MEDITECH, we downloaded the following data elements for all ED visits that occurred between January 1, 2010, and December 31, 2010: 1) demographic characteristics (age and sex), 2) clinical characteristics (chief complaint, triage level, and mode of arrival), 3) date and time of arrival, 4) date and time of room placement, 5) date and time of decision to admit to an inpatient or observation bed, 6) date and time of disposition, and 7) disposition status. The study ED uses the Emergency Severity Index to triage patients.12 The triage nurse documents each patient’s chief complaint using a standardized list in MEDITECH.
MEDITECH has a patient tracking system that automatically records the time when a staff member moves the patient from one status event to another. For example, when the patient is placed in a room, the treating nurse changes the status from “waiting for treatment room” to “in room.” The only activity status that is not captured by the tracker is the admission decision time. A member of the registration staff manually enters the admission decision time according to documentation provided by the attending physician on the admission request form for an inpatient or observation bed. In a previous study that used the same data elements, we examined the accuracy of the electronic patient tracking data for a small random sample of 275 ED visits and found that the electronic data accurately captured patients’ movement from one phase of care to the next.13
Based on the data, we calculated each patient’s waiting room time, treatment time, and boarding time, if admitted. Waiting room time was defined as the interval between arrival time and time of room placement. Treatment time was defined as the interval between room placement and physical departure from the ED for discharged patients. For admitted patients, treatment time was the interval between room placement and the admission decision time. Boarding time was defined as the period between the admission decision time and physical departure from the ED.
Crowding was measured using ED census. We calculated ED census as the total number of patients physically in the ED during a specified time interval. Using the total ED census distribution, we categorized crowding as low (<25th percentile), medium (25th–75th percentiles), and high (>75th percentile). This study was approved by the institutional review boards of both institutions as part of a career development grant awarded to the first author.
More Variation in Within-day Crowding Than Between-day Crowding
When measuring crowding, one of the first questions that must be addressed is how frequently should crowding be measured? If one measures crowding on a daily basis, is this sufficient to accurately capture the effect that crowding has on the outcomes of patients treated in the ED that day? In fact, there is more variation in crowding within a 24-hour period than across 24-hour periods. Crowding can double within a single day but rarely doubles from one day to another. To illustrate this point, we measured total ED census hourly for the 1-year study period (n = 8,760 hours). The median ED census for all 8,760 hours was 36. We divided the ED census of each study hour during the 1-year period by the median hourly ED census to create an hourly ED census ratio. The advantage of standardizing the individual values to the median is that the ratio has the same interpretation when measuring crowding across multiple EDs. When this ratio is equal to 1, it means that the ED census for a particular hour is equal to the median hourly ED census. When it is lower than 1, the hourly ED census is lower than the median, and when it is greater than 1, it is higher than the median. The top histogram in Figure 1 shows the percentage of hours that the ED census was various distances above or below the median. The variance of this hourly census ratio represented in the top histogram is 0.12.
We used this same approach to characterize the variation in crowding on a daily basis. We measured total ED census on a daily basis for the 365 days of the study period and divided daily ED census by the median daily ED census. The median daily census for the study ED was 161. The bottom histogram in Figure 1 shows the distribution of daily values relative to the median. As expected, Figure 1 illustrates that variation in crowding is larger hourly than daily. During the 1-year period, 9% of the hours had a total ED census at least 50% greater than the median hourly census. In contrast, for none of the days was the total ED census at least 50% greater than the median daily census. The variance of this distribution of daily ED census ratio is 0.01, 1/12 as large as the hourly distribution. As variance is a measure of information, the hourly variation has the potential to provide 12 times as much information, for example, about the effects of crowding on LOS or patient outcomes. The results of Figure 1 demonstrate that much of the variation in crowding will be undetected by daily measures.
Measuring Crowding Statically or Dynamically
A second important question is: how accurate is a single measure of crowding, whether it be at one time, or averaged over time, compared to measuring crowding multiple times throughout each patient’s ED stay? Figure 2 illustrates the total ED census measured hourly for 31 days in August 2010 at the study ED. On August 23, the line bolded in black shows that if a patient arrives at 8:00am and crowding is measured at the time of arrival, then the patient’s crowding value would be 22. If this patient was discharged at noon and crowding is measured by the average ED census during that patient’s 4-hour ED LOS, the crowding value would be 31. If crowding is measured by the mean census during the shift in which the patient arrived (8am–4pm), the crowding value (shift mean ED census) would be 42.
To compare the variation in crowding that is measured at a single time point or pooled over an interval, we measured total ED census every 15 minutes during the 1-year study period and compared these frequent measures of crowding to crowding measured at arrival as well as crowding averaged over the shift for 54,706 patients in the dataset (excluding those who left without being seen).
To do this, we transposed the ED visit data from a person-based file to a person-period file. We divided each patient’s LOS into 15-minute intervals and for each 15-minute interval that the patient remained in the ED, we created a period-specific record for that person and interval. Thus, the person-period file has multiple records per subject, one for each time period. Table 2 illustrates the conversion of the data set from person only to person-period. In the person-based file, the first subject had a total ED LOS of 133 minutes, so we created nine person-period records for this subject, one for each consecutive 15-minute interval. The person-based record shows that crowding, measured every 15 minutes (C1–C9) ranged from 21 at arrival to 60 during the first subject’s ED LOS. In the person-period file, we used dummy variables (D1–D9) to indicate each 15-minute time period and to match the values of the covariates to the appropriate time period. For example, crowding at arrival (C1) is recorded in the first time interval record (D1). Likewise, the time that each subject completed a stage of care is indicated by a 1 in the interval in which it occurred. For the first subject, completion of waiting room time is indicated in the D4 record, decision to admit is recorded in the D6 interval, and ED discharge is documented in the D9 record.
|Time Period||ED Census During Each Time Period||Sex||Time of Care Completion|
|C1||C2||C3||C4||C5||C6||C7||C8||C9||Waiting Room Time||Treatment Time||Boarding Time|
|D1||D2||D3||D4||D5||D6||D7||D8||D9||Sex||ED Census||Waiting Room Time||Treatment Time||Boarding Time|
Within each phase of care, we divided each individual crowding value by the median crowding value of all 15-minute intervals to create a time-varying ED census ratio. For the two static measures, we computed the ratio between each patient’s ED census at arrival and the overall median ED census at arrival, as well as the ratio between the mean shift ED census (based on the shift in which the patient arrived) and the study ED’s overall average shift ED census.
Figure 3 displays nine histograms of the distribution of the time-varying ED census ratio; ED census at arrival ratio; and the shift mean ED census ratio for waiting room time, treatment time, and boarding time. For all three phases of care (and for overall LOS), the time-dependent census and ED census at arrival show greater distances from the median compared to ED census averaged over the shift. For waiting room time, the variance is largest for ED census at arrival (0.10) compared to time-varying ED census (0.08) or mean shift census (0.07). In contrast, for treatment time and boarding time, the variance is approximately the same for time-varying ED census and ED census at arrival (0.10 and 0.09, respectively) and smaller for mean census during the shift (0.07).
In this comparison, ED census at arrival demonstrated similar variation in crowding as the time-varying crowding measure. This finding was unexpected and may be partially due to the fact that ED census does not vary as much by hour of day as compared to another crowding measure, such as hourly ED arrivals.14 The more hourly variation in the crowding measure, the more likely we would have seen greater differences between the dynamic and static measures. Despite the small differences between time-varying ED census and ED census at arrival, we recommend using the dynamic measure if possible because it measures the crowding exposure at different points in time during a patient’s LOS. While crowding may have a negative effect on all patients in the ED, it will likely affect some patients more than others depending on where patients are in their treatment trajectory and how crowded it is at the moments in time when they are competing for different resources.
Estimating the Effect of Crowding Using Linear or Logistic Regression Models
The vast majority of studies that have documented the effect of crowding on patient care and outcomes have measured crowding using linear or logistic regression models.15,16 Below is an equation for a linear regression model that shows the expected mean value (E) of an outcome (Y) given a set of predictors (X). The major differences between linear and logistic regression models are: 1) in linear regression, the expected value of the conditional mean can take on any value as x ranges from negative to positive infinity, whereas the expected value of the mean is bounded between 0 and 1 with a dichotomous outcome; and 2) the error term (ε) in logistic regression follows a binomial distribution rather than a normal distribution, as in linear regression.17 The simple equation below can be used to predict mean waiting room time (log-transformed since waiting room time is skewed) as a function of crowding, patient, and clinical characteristics:
Linear and logistic regression models that include crowding as a predictor are constrained in two important ways: 1) the exposure of crowding is represented by one measurement of crowding at a single time point or pooled across an interval as discussed earlier and 2) the model assumes that the effect of crowding on the outcome does not change over the course of time, such as during a patient’s LOS. In a linear or logistic regression model, there is usually only one parameter that estimates the effect of crowding on the outcome.
Estimating the Effect of Crowding Using a Discrete Time Survival Analysis Model
Another approach to estimating the effect of crowding on a dichotomous or continuous outcome is with a discrete time survival analysis model.18 This model is advantageous when an investigator is interested in determining not only if an outcome occurs, but when it occurs. The discrete time survival analysis model estimates the discrete time hazard (ht) or conditional probability that an individual will experience an outcome (Y) in time period t given that the individual did not experience the outcome in a prior time period: ht = Pr[Y = t|Y ≥ t].
We can plot the discrete time probability parameters ht against time (hazard function) as the population risk of the outcome occurring in time period t, conditional on the outcome not having occurred previously. This hazard function plot describes how the risk of an outcome occurring changes over time. In addition to estimating the baseline risk of an outcome occurring, the discrete time survival analysis model can also estimate how a risk profile changes by the presence of different predictors. To estimate the shift in baseline risk associated with a predictor, the hazard probabilities are reparameterized so that the model estimates the log odds of the outcome occurring as a function of different predictors. An example of a simple discrete time survival analysis model that estimates the odds of being placed in a treatment room (h) for a given individual (i) in a specific time period (t) for different crowding levels is
In a discrete time survival analysis model, time (t) is a predictor and is represented by multiple intercepts (D1–D9), one for each time period. When all predictors are equal to 0, these α intercepts estimate the odds of the outcome occurring (i.e., completion of treatment) over time. The β coefficients represent the relative odds associated with different predictor variables. The model above includes one time-varying predictor and four time-invariant predictors. Time-varying or time-dependent predictors are those whose values can change over time, such as crowding, staffing, availability of resources, etc. Time-invariant predictors are factors that remain constant throughout the study period, such as sex, acuity, and chief complaint.
The above model can be extended to include an interaction term of time × crowding to determine whether the effect of crowding on the odds of completing different phases of emergency care changes over time. Most studies to date have examined whether crowding affects the occurrence of an event within a given time period, such as within 4 hours of arrival or within 7 days of ED discharge. However, it is likely that the effect of crowding on the occurrence of an outcome varies over time (for example, greater earlier in the time period and less influential later on or vice versa). The discrete time survival analysis model allows one to evaluate whether the effect of crowding on an outcome is constant or varies over time.
Using the person-period data set we described in the previous section, Figure 4 illustrates that the impact of crowding on the odds of completing care changes over time. If an interaction term of crowding and time is not included in the discrete time survival analysis model, then the risk of completing a phase of care is assumed to remain the same throughout a patient’s stay and is represented by the dashed lines in each of the graphs. However, when an interaction term is included, it is clear that the effect of crowding on completion of care changes over time. For example, in the waiting room time model, the relative odds of completing waiting room time during highly crowded periods versus low crowding periods is highest at arrival and decreases the longer the patient is in the ED. In contrast, for boarding, the odds of completing boarding is initially lower for patients who are in the ED during high versus low crowding periods, but increases steadily the longer they wait for an inpatient bed.
To directly compare crowding results estimated from a discrete time survival analysis model to those from a linear or logistic regression model is difficult because of the fundamental differences among the models. The discrete time survival analysis model can estimate the effect of crowding dynamically, whereas the linear and logistic regression cannot. However, they can both estimate the hazard function or the conditional probability of completing a phase of care at each time interval by different crowding levels. In the linear regression model, we log-transformed each outcome (i.e., waiting room time, treatment time, and boarding time) and used the normal assumption on the log scale, which corresponds to a log-normal distribution to estimate the service completion times. For the discrete time survival analysis model, the hazard function is estimated from the conditional probabilities of completing care in each time interval.
In Figure 5, we have plotted the observed and predicted hazard of completing waiting room time, treatment time, and boarding time by the different models (i.e., discrete time survival analysis vs. linear regression) and crowding levels. Crowding has the biggest effect on waiting room time, and patients who are in the ED during high crowding periods have a lower probability of being placed in a room compared to those in the ED during low crowding periods, regardless of the model used. In addition, the predicted probabilities estimated from the discrete time survival analysis model fit the observed data better than the estimates from the linear regression models that measure crowding either at time of arrival or as a shift average. The discrete time survival analysis model is more flexible to describe the actual hazard rather than imposing a parametric assumption such as the lognormal distribution. It also allows for the inclusion of an interaction term of crowding and time compared to the linear regression model, which does not because it cannot accommodate dynamic predictors. The linear regression models are frequently underestimating the probability of completion of care. For waiting room time, this only occurs during low crowding periods. For treatment time and boarding time, the linear regression models initially overestimate the rate of completion of care for the first 1 to 2 hours and then underestimate it thereafter.
The methods used to measure crowding will affect the inferences one makes about the relationship between crowding and an outcome. First, the more granular the measurements of crowding, the more information is preserved to study crowding effects on health outcomes. In particular, crowding that is measured at the daily level will mask much of the variation in crowding that occurs within a 24-hour period. Second, of the two static measures, ED census at arrival demonstrates more variation in crowding compared to mean ED census during the shift. ED census at arrival and time-varying ED census demonstrated similar variation in crowding exposure on ED LOS. Finally, the discrete time survival analysis model is a preferred, more flexible approach to estimating the effect of crowding on an outcome because it allows one to include time-varying predictors and to evaluate whether the influence of different predictors on an outcome is constant or changes over time.
- 3General Accounting Office. Hospital Emergency Departments - Crowded Conditions Vary Among Hospitals and Communities. Washington, DC: United States General Accounting Office; 2003. Report No.: GAO-03-460.
- 17Applied Logistic Regression. New York, NY: John Wiley & Sons, 1989., .
- 18Survival Analysis: Techniques for Censored and Truncated Data. 2nd ed. New York, NY: Springer-Verlag, 2009., .