This study was funded by an internal grant from the University of Louisville, Department of Pediatrics (#GR1719-CROS2). None of the authors have financial interests related to any of the topics discussed in this paper.
Identifying Key Metrics for Reducing Premature Departure from the Pediatric Emergency Department
Version of Record online: 2 NOV 2010
© 2010 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 17, Issue 11, pages 1197–1206, November 2010
How to Cite
Cross, K. P., Gracely, E., Stevenson, M. D. and Woods, C. R. (2010), Identifying Key Metrics for Reducing Premature Departure from the Pediatric Emergency Department. Academic Emergency Medicine, 17: 1197–1206. doi: 10.1111/j.1553-2712.2010.00908.x
Supervising Editor: Marc Gorelick, MD.
- Issue online: 2 NOV 2010
- Version of Record online: 2 NOV 2010
- Received February 20, 2010; revision received April 13, 2010; accepted April 14, 2010.
Objectives: Approximately 2% to 5% of children presenting to pediatric emergency departments (PEDs) leave prior to a complete evaluation. This study assessed risk factors for premature departure (PD) from a PED to identify key metrics and cutoffs for reducing the PD rate.
Methods: A 3-year cohort (June 2004–May 2007) of children presenting to a PED was evaluated. Children were excluded if they presented for psychiatric issues, were held awaiting hospital admission in the PED due to a lack of inpatient beds, were more than 21 years old, or died before disposition. Univariate analyses, multivariable logistic regression, and recursive partitioning were used to identify factors associated with PD. A fourth year of data (June 2007–May 2008) was used for validation and sensitivity analysis.
Results: There were 132,324 patient visits in the 3-year derivation data set with a 3.8% PD rate, and 45,001 visits in the fourth-year validation data set with a 4.3% PD rate. PDs were minimized when average wait time was below 110 minutes, concurrent PDs were fewer than two, and average length of stay (LOS) was less than 224 minutes in the derivation set, with similar results in the validation set. When these metrics were exceeded, PD rates were over 10% among low-acuity patients. These findings were robust across a broad range of assumptions during sensitivity analysis.
Conclusions: The authors identified five key metrics associated with PD in the PED: average wait time, average LOS, acuity, concurrent PDs, and arrival rate. Operational cutoffs for these metrics, determined by recursive partitioning, may be useful to physicians and administrators when selecting specific interventions to address PDs from the PED.
ACADEMIC EMERGENCY MEDICINE 2010; 17:1197–1206 © 2010 by the Society for Academic Emergency Medicine
A premature departure (PD)—leaving the emergency department (ED) before full medical evaluation and routine disposition—raises considerable ethical, medical–legal, financial, customer satisfaction, and patient safety concerns, particularly for pediatric patients. Furthermore, rising volume in EDs has led to increases in the rate of PD.1–5
Premature departure from a pediatric emergency department (PED) can happen at a variety of points during a typical visit. There have been numerous studies of various forms of PD in adult, pediatric (PED), and mixed ED settings. Reported PD rates have ranged from 0.1%6 to 15%,7,8 with most falling between 2% and 5%.4,9–14 The PD rate from PEDs in a national sample in 2005 was 5.73%.13
Several issues limit understanding which factors contribute to patients departing prematurely from the PED. The majority of studies on this topic have not focused on pediatric settings, and the importance of various factors may vary depending on patient age and family considerations. For example, one study found time of arrival and race to be significant predictors of PD among adults, but not among children.13
When pediatric settings have been studied, there have been methodology limitations. Five of seven previous studies of pediatric PD used post hoc survey methods, which have significant selection and recall biases.1,3,10,13–16 Differences in methods have yielded conflicting results, such as arrival time being significant in some studies,10,14,15 but not others.13
Risk factors for pediatric PD suggested in these prior studies have included arrival time, acuity, rate of arrivals, concurrent PDs, individual patient wait time, availability of a primary care provider, average wait time, average length of stay (LOS), arrival day of the week, arrival month of the year, hospital urban location, payer, race, and various markers of socioeconomic status (SES).1,3,10,13–16
Although a variety of risk factors appear associated with PD in these studies, specific evidence-based interventions to reduce PD rates have not been fully explored.17 Knowing that a factor is associated with PD does little to guide operational changes unless the following questions can be answered:
- • Which real-time metrics should be monitored?
- • What levels for those monitored metrics are most concerning?
- • Which sentinel events should trigger increased staffing?
- • What return would be expected on investments to reduce PD rates?
- • What mix of increased staffing and skill sets is needed?
- • Which PED processes would benefit from reengineering?
Specifically, our study examines the first three questions and suggests areas for future research in the remainder. We conducted this study to explore factors associated with PD from the PED and to identify specific operational cutoffs for key performance metrics that could guide development of evidence-based interventions to lower the rate of PD.
This project was a retrospective cohort study. The study was approved by the local institutional review board with a full waiver of informed consent; no personally identifying patient information was used.
Study Setting and Population
All qualifying PED visits over a 4-year period at a single tertiary free-standing children’s hospital with an annual volume of about 50,000 visits were included. The hospital is a regional trauma center and has the only PED in a 115-km radius. It serves a catchment area with a population of approximately 2 million people. This study built upon our previously published pilot work done on this topic.14
Patients aged less than 21 years (which is the upper limit for routine PED visits at our institution) were included in the study if they presented to the PED and registered with the greeter to be seen. At that time, a record is created in the PED patient tracking system EMSTAT (Allscripts, Inc., Austin, TX). Patients were excluded from the study if they presented to the PED for laboratory or radiology studies only, if their chief complaint was psychiatric in nature, if they died, if they had already been seen at another site and were presenting only for direct admission to the hospital, or if they were transferred to another facility for care (e.g., pregnant patients sent to another hospital for obstetric services). Patients meeting these criteria were excluded because they do not follow a typical process flow through the PED or because they have limited ability to leave prematurely (e.g., psychiatric patients). Patients were also excluded if they were held in the PED overnight due to unavailability of inpatient beds. Their process flow is atypical and their metrics (e.g., LOS) do not reflect PED processing. These “held” patients were identified electronically in EMSTAT as recorded by the patient care team at the time of service.
Data records were extracted from June 1, 2004, through May 31, 2008, from EMSTAT and downloaded into Excel (Microsoft Corp., Redmond, WA) for data formatting and assignment of SES information. SES information was assigned based on 2000 U.S. census data by zip code. Only patients with recorded zip codes could be assigned SES information. Only patients from zip codes with at least 1,000 visits were included in the SES assignment process. We included the following SES attributes: median household income (in U.S. dollars), high school graduation rate (%), nonwhite race rate (%), non–English-speaking at home (%), and poverty rate (%). This method of assigning SES data imputed from a patient’s home zip code has been used elsewhere in the evaluation of PD.14,18
Data records were separated into two data sets for analysis using SPSS version 18 (SPSS Inc., Chicago, IL). The first data set contained all records meeting inclusion and exclusion criteria from June 1, 2004, through May 31, 2007, and is subsequently referred to as “Years 1–3.” The second data set contained records from June 1, 2007, through May 31, 2008, referred to as “Year 4.” This split was chosen because at that time our PED changed from a four-tier triage system (critical, urgent, near-urgent, nonurgent) to a new five-tier system based on the Emergency Severity Index.19 There was no clear one-to-one mapping of patient acuities from the four-tier system to the five-tier system. Because acuity has been a strong predictor of PDs in prior studies,4,6,10,12–15,20–22 we kept the two data sets separate. Unless specifically stated, all analyses presented used the Years 1–3 data set. All acuity assignments are presented in this study using the following numbering scheme: critical—1; emergent—2 (Year 4 data only); urgent—3; near-urgent—4; nonurgent—5.
The primary outcome for this study was PD, defined as any PED visit that met study inclusion and exclusion criteria without a routine disposition. A routine disposition meant the patient completed evaluation and treatment in the PED as planned by the medical staff. Routine disposition patients were admitted or discharged, while PD patients either left without being seen, left after being seen, or left against medical advice. These results were determined from the disposition recorded in EMSTAT. The PD rate was defined as the number of patients who departed prematurely in a given time period divided by all patients meeting inclusion or exclusion criteria in that period and is expressed as a percentage.
Outcomes from recursive partitioning analyses are presented as partitioning trees, with information on the percentage of correctly classified subjects and the areas under their receiver operator characteristic curves (AUCs).
For both the Years 1–3 data set and the Year 4 data set, a variety of assigned and calculated metrics were constructed in SPSS for each patient. Assigned metrics included travel distance and monthly volume. Calculated metrics included individual patient LOS, individual patient wait time, average LOS, average wait time, arrival rate, and concurrent PDs.
Travel distance was assigned to patients based on their home zip codes. Patients were divided into two groups: those whose home zip codes were predominantly within 8 km (5 miles) of the hospital and those further than 8 km.
Monthly volume was the sum of all included patient visits for each month and was assigned to each patient on the basis of his or her arrival month.
The individual patient LOS for each patient was the difference in minutes between the patient’s arrival and disposition times. The individual patient wait time was the difference in minutes between their arrival time and the time a physician (advanced practice nurse, resident, or attending physician) acknowledged self-assignment to the patient on the EMSTAT system, which typically happened when the provider picked up the chart to see the patient. For patients who left before being seen, the individual wait time equaled the individual LOS.
Time series analysis in SPSS was used for several additional calculated metrics that reflect general conditions in the PED at the time a given patient arrived for care, as described in our previous analysis.14
Average LOS and average wait time were assigned to each patient in this manner: for each patient, the average LOS was the rolling average of individual patient LOS values for the 20 patients who presented to the PED immediately prior to the current patient. Similarly, the average wait time was the rolling average of individual patient wait times for the 20 patients presenting prior to the current patient. A 20-patient window was chosen for this metric because work during our previous pilot study showed this window to be most associated with PDs.14
Arrival rate assigned to a given patient was the estimated rate of arrivals in patients per hour at the time of presentation. It was calculated from the inverse of the interarrival time for the preceding 10 patients using the following formula: (10 × 60) /IAT10 where IAT10 is the interarrival time in minutes between the current patient and the 10th prior patient.14 The constant 60 is in units of minutes per hour, and the constant 10 is in units of patients. The metric describes for each patient the number of patients who would arrive in an hour, if the arrival rate of the preceding 10 patients were sustained for the whole hour. A window of 10 patients was chosen to reflect typical arrival rate per hour seen at our institution during busy periods.
The concurrent PDs metric for each patient was the number of PDs occurring among the preceding 40 patients presenting to the PED. A 40-patient window was arbitrarily chosen for this metric because it is our average volume in one 8-hour shift (roughly 120 patients per day divided by three shifts).
Univariate and multivariate analyses were performed using SPSS. Continuous variables were analyzed with t-tests or Mann-Whitney U-tests, while discrete variables were analyzed with chi-square and Fisher’s exact tests. Factors with p-values less than 0.2 in univariate analysis were subsequently selected for inclusion in both the regression and the recursive partitioning analyses as discussed below.
Binary logistic regression with backward conditional selection was used for multivariate analysis. Sensitivity analysis for regression included four different models: the first had a limited set of factors but included virtually all patients, while the fourth had the broadest set of factors but only included a smaller set of patients with sufficiently complete information for analysis.
Recursive partitioning was used to evaluate complex relationships and find highly discriminating risk factors. Recursive partitioning was performed using CART software (Salford Systems, San Diego, CA). The Years 1–3 data set was used for derivation and the Year 4 data set for validation. Eighteen independent variables used in logistic regression analysis were also included in recursive partitioning. A 19th variable (individual patient wait time) was added during sensitivity analyses. Sensitivity analyses were performed by varying the data used for derivation (Years 1–3 data set, Year 4 data set, combining both data sets, etc.) and by varying the processing methods used (minimum node sizes, maximum tree depth, missing variable handling logic, etc.).
The accuracies of different models (trees) generated by recursive partitioning were assessed by the AUC. A perfect classification scheme would have AUC = 1.0, indicating that for each criterion in the pathway, there was 100% sensitivity for all PD cases and 100% specificity for all non-PD cases. An AUC > 0.5 indicates the tree is more discriminating than chance alone.
There were 185,280 patient visit records on the source database for the 4-year study period. As shown in Figure 1, there were 7,937 records that met exclusion criteria. The remaining 177,343 records were split into a derivation data set (Years 1–3) of 132,342 records, including 5,091 PDs, and a validation and sensitivity analysis data set (Year 4) of 45,001 records, including 1,919 PDs. PD rates were 3.8 and 4.3%, respectively. Approximately 89% of PD patients left without being seen (6,215/7,015).
Univariate analysis showed a wide variety of factors that appeared to be associated with PDs (p < 0.2), which were then included in multivariate modeling of the Years 1–3 data set (data not shown). The only factor that was not significant in univariate analysis (p > 0.2) was the SES marker for non–English-speaking households, and it was not included in multivariate analysis.
Results for multivariate logistic regression for a base case (Model I) and several subsequent cases of increasing complexity (Models II, III, and IV) are summarized in Table 1. The following factors were found to have statistically significant association with increasing PD rates: arrival time of day, arrival day of the week, arrival month, arrival year, lower acuity, higher concurrent PDs, higher arrival rate, longer average wait time and average LOS, public payers or self-payers, living in a zip code with lower median household income or higher poverty rate, and longer individual patient wait time.
|Model I||Model II||Model III||Model IV|
|Hosmer-Lemeshow goodness of fit||χ2 = 110.1 p < 0.001||χ2 = 92.8 p < 0.001||χ2 = 69.0 p < 0.001||χ2 = 90.0 p < 0.001|
|Significant independent factors||Arrival time Arrival month Acuity Concurrent PDs Arrival rate Average LOS Arrival day Arrival year Average wait||Arrival time Arrival month Acuity Concurrent PDs Arrival rate Average LOS Arrival day Arrival year Average wait Payer||Arrival time Arrival month Acuity Concurrent PDs Arrival rate Average LOS Arrival day Arrival year Average wait Payer SES income SES poverty||Arrival time Arrival month Acuity Concurrent PDs Arrival rate Average LOS Arrival day Arrival year Average wait Payer SES income SES poverty Individual wait time|
|Nonsignificant factors||Age Monthly volume||Age Monthly volume Travel distance Captured race||Age Monthly volume Travel distance Captured race SES race SES graduation||Age Monthly volume Travel distance Captured race SES race SES graduation|
The following factors were consistently nonsignificant in all multivariate models: patient age, monthly volume, travel distance, race (as recorded during registration), and living in a zip code with higher or lower percentage of nonwhite race or high school graduation rates. Notably, arrival month was significant, but monthly volume was not significant in this analysis, suggesting that arrival month may represent more discriminating information to the model than just volume alone—such as seasonality of different types of illness or injury, school and holiday calendar effects, and house officer training cycles.
Hosmer-Lemeshow goodness-of-fit tests showed that improvements in fit were possible for all four regression models (Table 1). One common source of Hosmer-Lemeshow poor fit is interactions among the modeled variables. To address the possibility of interactions and to help define useful cut-points for classification, we used recursive partitioning, as described in the next section.
Identification of Operational Metrics and Discriminatory Breakpoints
Recursive partitioning was used to further evaluate variables associated with PD for operational importance and to identify breakpoints or levels of events that could be used for process improvement. The initial (base case) recursive partitioning used the Years 1–3 records as the derivation data set and the Year 4 records as the validation data set. Candidate predictor variables for the base case were identical to those included in regression Model III (see Table 1 above). The only candidate variable not included in this base case analysis was individual patient wait time, which can be both a cause and a result of the decision to leave prematurely and therefore was not considered a truly independent variable. Individual patient wait time was subsequently included in sensitivity analysis, however.
The least complex tree within one standard error of optimal performance was selected as the base case result and is shown in Figure 2. The AUC for this tree was 0.79 for derivation and 0.69 for validation. This tree correctly classified 78.1% of 5,096 PD cases and 71.7% of 127,246 non-PD cases in the derivation data set. In the validation data set, this tree correctly classified 60.8% of 1,919 PD cases and 74.8% of 43,082 non-PD cases.
Predictors of increased PD rate were average wait time greater than 110 minutes and acuity of 4 or 5 (low acuity). Among low-acuity patients, an arrival rate greater than three patients per hour was associated with PDs whenever the average wait time was more than 110 minutes. When the average wait time was less than 110 minutes, patients were more likely to depart prematurely when the average LOS was greater than 224 minutes, the arrival rate was greater than six patients per hour, and there was more than one concurrent PD.
Sensitivity analysis was performed using CART software for over 300 different combinations of data sets, processing logic, and validation approaches. One simple sensitivity analysis was reversing the derivation and validation data sets; that is, using Year 4 as the derivation dataset and Years 1–3 as the validation data set (Figure 3). For this tree, the AUC was 0.78 for derivation and 0.74 for validation. In the derivation data set, this tree correctly classified 77.5% of 1,919 PD cases, and 72.6% of 43,082 non-PD cases. This tree correctly classified 78.0% of 5,096 PD cases and 68.9% of 127,246 non-PD cases in the validation data set.
Note that while consistent at the upper levels of the tree, the farther down the tree one explores, the more divergence is seen between Figure 2 and Figure 3. For example, average LOS, which is a third-level splitter in Figure 2, does not appear in Figure 3. Instead, it is replaced by another, tighter cutoff for average wait time. In general, the farther down a tree we explored during any given sensitivity analysis, the more divergence from the base case in Figure 2 was apparent, but the less effect was attributable to any specific finding. The top of the tree, however, remained robust.
Sensitivity analysis included variations in the splitting rules to select partition variables, the logic to handle missing data, the validation methods, the maximum tree depth, and the minimum node size. It also used different subsets of data: Years 1–3 data set, Year 4 data set, all years combined, serial removal of predictor variables, and substitution of target variables.
Across different combinations of assumptions, methods, and data sets, sensitivity analyses showed the same important predictor variables dominant in most solutions, typically with cutoffs at or quite near the ones presented in Figures 2 and 3 (sensitivity analysis data not shown). Excluding patients who left against medical advice had no significant effect on the results. Changing the validation method from use of an alternate dataset (Years 1–3 rules validated against Year 4 data) to use of V-fold validation algorithms (Year 1–3 rules validated against random subsets of Year 1–3 data) did not materially change the results either.
Arrival rate appears in several places in these partitioning trees with different cutoff values. To facilitate understanding of its magnitude at our institution, Figure 4 shows the median arrival rate, with 5th and 95th percentiles, by hour of the day.
What return would be expected on investments to reduce PD rates? Figure 2 suggests an answer to this question. It shows a PD rate of 8.1% for the 44,215 patients who presented when average wait times were longer than 110 minutes (the right branch of the tree). Any intervention that keeps average wait times below 110 minutes might convert these patients to the left branch of the tree, with a PD rate of 1.7%. With such an intervention, there might be approximately (8.1%− 1.7%) × 44,215 = 2,830 fewer PDs over 3 years. Therefore, the expected gain in revenue from 2,830 more paying visits would be offset against the cost of any intervention(s) that lower average wait time below 110 minutes. This pro forma analysis can be repeated at other levels of the tree in Figure 2 to forecast likely return of any targeted investment applied to the problem of PDs.
What mix of increased staffing and skill sets is needed? The information from Figure 2 suggests that the areas of greatest impact are low-acuity, long-wait patients. On both sides of the tree, it is acuity levels 4 and 5 that have high (potentially fixable) PD rates. Therefore, low-acuity resources will likely have the largest impact on PD rates when adding resources in the studied setting. For example, high-skill providers (e.g., more pediatric emergency attendings) may not be as cost-effective as adding nurse practitioners or general pediatricians, who can manage low-acuity patients. Further research regarding the efficacy of additional resources to facilitate evaluation and treatment of low-acuity patients in reducing PD is warranted.
Which PED processes would benefit from reengineering? Again, Figure 2 strongly suggests that low-acuity processes are in greatest need of reengineering for the studied setting. Reengineering processes for high-acuity patients will probably result in little improvement in PD rates (which are already quite low, even when average wait times are long). The recursive partitioning methods used in this study can be applied to similar data sets from other institutions to determine operational metrics and cutoffs and to suggest interventions that may reduce PD rates at those facilities.
A summary of the most robust findings associated with increased rate of PD is displayed in Table 2, along with associated operational goals and proposed interventions developed with our institution’s management staff. Many of the interventions suggested in this table have appeared elsewhere in the literature.1,3,10,13,15,17
|PD Risk Factors||Associated Operational Goals||Possible Interventions|
|Average wait > 86, 110, or 120 minutes||Keep average wait times below 110 minutes and if possible below 86 minutes.||• Increase staffing during periods known to have long wait times. • Decrease staffing when wait times are short. • Reengineer triage process to shorten time-to-physician periods. • Use “on-call” teams to augment throughput during high-volume periods.|
|Acuity = (3), 4, 5 (i.e., low acuity)||Focus resources and process reengineering on low-acuity “fast track” areas.||• When reengineering operations, start with low-acuity processes. • Deploy extra resources preferentially to low-acuity areas.|
|Average LOS > 224 minutes||Once wait times are minimized, reducing total LOS becomes the focus.||• Initiate treatments in the waiting/triage areas (e.g., pain medications, simple radiographs, antiemetics, antipyretics). • Use point-of-care tests when possible to avoid laboratory turnaround time delays. • Use metered dose inhalers instead of nebulizers for short dose albuterol. • Use bedside ultrasound where applicable to avoid radiology studies (e.g., FAST exams, fracture diagnosis). • Avoid “gridlock” when there are no beds available to see additional patients (e.g., stable patients awaiting laboratory results return to the waiting area, admitted patients moved to inpatient wards promptly).|
|Concurrent PDs > 1||Treat PDs as sentinel events that trigger real-time response. Minimize waiting room exposure to other dissatisfied families.||• Bring in extra “on-call” staffing when CPDs exceed one during a shift. • Decompress the main waiting areas by moving families to “overflow” treatment areas (even if they then wait there). • Frequently announce emergency department status and wait times. • Initiate treatment in the waiting and triage areas.|
|Arrival rate > 3, 4, or 6 patients per hour||Real-time monitoring of arrival rate (along with CPDs) used as an escalation trigger.||• Set trigger points for arrival rate, wait times and concurrent PDs for “on-call” resources. • Escalate resources in proportion to how much target metrics are exceeded.|
Premature departure of patients from EDs results in missed opportunities to treat suffering and prevent the spread of disease, low patient satisfaction, lost revenue to providers, and increased medicolegal risks.3,15,23 We used a large database of unselected PED visits to identify and confirm risk factors associated with PD and to characterize key cutoff points at which they become significant. These cutoffs may be used as operational goals and real-time triggers of operational interventions.
In our study, the most discriminating predictor of pediatric PD was the average wait time at the moment a given patient arrives in the PED. An average wait time longer than 110 to 120 minutes was strongly associated with increased PD risk across a wide range of sensitivity analyses. It was more predictive than even the individual patient’s own wait time (data not shown) during sensitivity analysis, in part because both long and short individual wait times are associated with PD (e.g., people who see a very crowded waiting room and rapidly decide to leave have a short individual wait time). Other studies have specifically examined average wait time (as opposed to individual wait times) and also found it to be strongly correlated with PD.11,14
Several other studies of pediatric PDs found low acuity to be a key risk factor.1,10,13–15 The combination of low acuity and long average wait time was particularly toxic. In our study, both recursive partitioning trees identified subsets of low-acuity patients who arrived when the average wait time was long. These low-acuity subsets had PD rates greater than 10%. Additionally, when the arrival rate was also high, the PD rate rose further to 11.8 and 13.2%. Particularly high-risk subsets like these are easily missed when only overall daily or monthly PD rates are reviewed; indeed, these results were surprising to the investigators and the managerial staff at our institution. These results strongly suggest that when adding resources or reengineering processes, investments in low-acuity operations may return the greatest improvement in PD rate.
Average LOS was another important predictor of pediatric PD in our study, with a cutoff time of 224 minutes. As with average wait time, this metric is associated with PDs in several other studies.10,12,14 Average LOS was important in the Year 1–3 data set, but did not appear as a key factor in some sensitivity analyses (e.g., Year 4 data as shown in Figure 3). Average LOS may best be regarded as a secondary operational target to be improved once other issues have been addressed.
Concurrent PDs is a novel, intuitive metric that we recently identified.14 If families in a PED waiting area see lots of other families leaving prematurely, they may be more likely to follow suit. Note that this metric appears in the base case tree (see Figure 2), but was not a significant variable in some sensitivity analyses (see Figure 3). While not the most discriminating metric among our results, it has the advantage of being a potential trigger or “sentinel event” during busy times in the PED. Our results suggest that more than one PD during a shift potentially indicates that others will follow.
Arrival rate has been associated with PDs in other studies.2,4,14,20 In our study, it appeared in several places in the partitioning trees. When average wait time was high, patients with low acuity were already at high PD risk, and even a relatively low arrival rate breakpoint (fewer than three patients per hour) conferred higher risk. Conversely, when PD risks were otherwise low, arrival rate breakpoints were higher (four or six patients per hour).
The results from this analysis also help answer three operationally important questions not directly studied:
This study analyzed data from a single site and a “typical” process flow for this institution, which potentially limits its generalizability. We excluded some classes of patients, including psychiatric patients, patients older than 21 years, and patients held in the PED overnight for lack of inpatient beds, which may also limit generalizability. However, excluded patients of all types totaled only 4.3% of the records extracted for this study. The study is also subject to limitations common to retrospective cohort analyses: recording errors and biases, lack of randomization, and incomplete data. Recording information in EMSTAT is not a perfect reflection of reality. For example, LOS and wait time data for patients who left without being seen are commonly overstated because their departure time is typically logged only after they fail to answer multiple overhead calls in the waiting area by the PED staff.
Year 4 data reflect our institutional change to the five-tier Emergency Severity Index system from the prior four-tier system used in Years 1–3. Results from our derivation and validation must be interpreted in light of this change. In all cases, the lowest two acuity levels (4 and 5) partitioned together. However, acuity level 3 sometimes partitioned with “high” acuity, and sometimes with “low” acuity, depending on whether the derivation data set contained four-tier or five-tier acuity. Last, data needed to evaluate the effect of fluctuations in staffing levels, the mix of physician training, or the usefulness of several capacity metrics (bed occupancy rates, work indices, overcrowding scales, etc.) proposed elsewhere20,24–34 were not available to this study.
This study used recursive partitioning of available electronic data to identify specific operational metrics based on factors associated with premature departures from a pediatric ED. In our system, the key targets for these metrics include keeping average wait times less than 110 minutes, average length of stay less than 224 minutes, and fewer than two concurrent premature departures. The most effective interventions to improve the rate of premature departure likely should focus on throughput capacity for low-acuity patients when these targets are exceeded and the arrival rate exceeds three patients per hour. Further research should focus on evaluating the efficacy of targeted interventions to reduce the rate of premature departure in the pediatric ED.
The authors thank Dr. Gerard Rabalais and the Department of Pediatrics at the University of Louisville for funding this project. Scott Belin provided technical assistance with the data download process from the source EMSTAT database. We thank Merlyn Viray from Salford Systems for assistance with the CART software. Kendra Sikes provided compliance assistance and human subjects’ protection guidance to the study. Dr. Ruth Carrico provided methodology review and advice. Dr. Aaron Calhoun contributed to the conceptual design for an early phase of this study.
- 6Emergency department patients who leave without being seen by a doctor: the experience of a medical center in northern Taiwan. Chang Gung Med J. 2002; 25:367–73., , , , , .