Presented at the American Geriatric Society National Meeting, Chicago, IL, April 2009; and at the Society for Academic Emergency Medicine annual meeting in New Orleans, LA, May 2009.
Accuracy of the Emergency Severity Index Triage Instrument for Identifying Elder Emergency Department Patients Receiving an Immediate Life-saving Intervention
Article first published online: 1 MAR 2010
© 2010 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 17, Issue 3, pages 238–243, March 2010
How to Cite
Platts-Mills, T. F., Travers, D., Biese, K., McCall, B., Kizer, S., LaMantia, M., Busby-Whitehead, J. and Cairns, C. B. (2010), Accuracy of the Emergency Severity Index Triage Instrument for Identifying Elder Emergency Department Patients Receiving an Immediate Life-saving Intervention. Academic Emergency Medicine, 17: 238–243. doi: 10.1111/j.1553-2712.2010.00670.x
- Issue published online: 1 MAR 2010
- Article first published online: 1 MAR 2010
- Received August 31, 2009; revision received October 6, 2009; accepted October 7, 2009.
- emergency treatment
Objectives: The study objective was to determine the sensitivity and specificity of the Emergency Severity Index (ESI) triage instrument for the identification of elder patients receiving an immediate life-saving intervention in the emergency department (ED).
Methods: The authors reviewed medical records for consecutive patients 65 years or older who presented to a single academic ED serving a large community of elders during a 1-month period. ESI triage scores were compared to actual ED course with attention to the occurrence of an immediate life-saving intervention. The sensitivity and specificity of an ESI triage level of 1 for the identification of patients receiving an immediate intervention was calculated. For 50 cases, the triage nurse ESI designation was compared to the triage level determined by an expert triage nurse based on retrospective record review.
Results: Of 782 consecutive patients 65 years or older who presented to the ED, 18 (2%) had an ESI level of 1, 176 (23%) had an ESI level of 2, 461 (60%) had an ESI level of 3, 100 (13%) had an ESI level of 4, and 18 (2%) had an ESI level of 5. Twenty-six patients received an immediate life-saving intervention. ESI triage scores for these 26 individuals were as follows: ESI 1, 11 patients; ESI 2, nine patients; and ESI 3, six patients. The sensitivity of ESI to identify patients receiving an immediate intervention was 42.3% (95% confidence interval [CI] = 23.3% to 61.3%); the specificity was 99.2% (95% CI = 98.0% to 99.7%). For 17 of 50 cases in which actual triage nurse and expert nurse ESI levels disagreed, undertriage by the triage nurses was more common than overtriage (13 vs. 4 patients).
Conclusions: The ESI triage instrument identified fewer than half of elder patients receiving an immediate life-saving intervention. Failure to follow established ESI guidelines in the triage of elder patients may contribute to apparent undertriage.
ACADEMIC EMERGENCY MEDICINE 2010; 17:238–243 © 2010 by the Society for Academic Emergency Medicine
The annual rate of emergency department (ED) visits for elder patients is increasing.1–3 A descriptive study of a national database found a 34% increase in ED visits between 1993 and 2003 for patients aged 65 to 74 years and predicted a further doubling in ED visits by this population by 2013.1 Concurrent with this increase in ED visits by elders is an increase in ED crowding, resulting from both a nationwide increase in ED visits and a reduction in the total number of EDs.2,4–6 Elder patients are an important subset of ED patients because they have a higher percentage of severe illness and injury than other age groups of patients, as evidenced by higher rates of hospital and intensive care unit (ICU) admission.3,7 Because of the large number of ED visits by elders, the high proportion of acute and severe illness among elders, and the growing problem of ED crowding, accurate triage of elder patients is a key component to providing timely emergency care for this vulnerable population.
Existing data on the triage of elder ED patients demonstrate mixed performance, with some triage tools appearing to predict outcomes better than others. The Emergency Severity Index (ESI) is a widely used five-level triage instrument that has been shown to be reliable and valid in both pediatric and adult populations for identifying ED resource use and hospital admission.8–12 A single study of the triage of elders using the ESI has demonstrated that it is a valid predictor of hospitalization, ED length of stay, ED resource use, and survival at 1 year.12 In contrast, a study of the five-question Triage Risk Screening Tool found it to only weakly predict for ED revisit, hospitalization, or long-term care placement for patients 65 years or older.13 Although matching patients with resource needs is an important function of a triage system, the other essential function is to identify patients with acute severe illness or injury who are in need of an immediate life-saving intervention. To the best of our knowledge, no published study has evaluated the ability of ESI or any other triage instrument to identify elders in need of an immediate life-saving intervention.
The objective of this study was to assess the sensitivity and specificity of the ESI version 4 triage instrument in predicting the need for an immediate life-saving intervention among elder ED patients. Additionally, we sought to determine if mistriage of patients receiving an immediate intervention was due to misapplication of existing criteria or to lack of accuracy of the criteria themselves.
We conducted an observational study of consecutive patients 65 years or older presenting to a single academic ED during a 1-month period to determine the accuracy of ESI Level 1 for identifying patients who receive an immediate life-saving intervention. We compared ESI triage designations made by triage nurses to actual ED course to calculate the sensitivity and specificity of ESI Level 1. Additionally, all patients who were admitted to the ICU or died within 24 hours were reviewed by a second study author (TPM) to identify patients who should have received an immediate intervention but did not. For a subset of patients, triage nurse ESI designations were compared to expert triage nurse review of the triage portion of the patient’s chart. The study received approval from the hospital’s institutional review board with a waiver of informed consent.
Study Setting and Population
The study site is a single ED at a Level 1 trauma center, with an emergency medicine residency program and an annual census of approximately 64,000 patients. The ED is located in a city of 50,000 people with a large surrounding population of elders. It is also a tertiary referral center for the state of North Carolina. The study enrollment period was June 20 to July 20, 2008. All patients 65 years of age or older who presented to the ED during the study period were included in the assessment of whether patients received an immediate intervention. This included patients who came by emergency medical services transport or by private transport. However, patients who did not have a triage level recorded in the ED record were excluded from the calculations of the sensitivity and specificity of the ESI triage instrument.
Triage designations were made by one of 31 triage nurses who performed triage duties during the study period. Each of these triages nurses have worked full time as a nurse in the ED at our institution for 1 year prior to working at triage, and each have had formal training in the use of ESI criteria. Additionally, the ESI triage algorithm is present on the wall of each triage room. ESI has been the exclusive triage tool at our institution since its introduction in 1999. Triage nurses were not informed that a study was being performed.
Data were abstracted using a computer-based abstraction form (Microsoft Excel 2007, Microsoft Corp., Redmond, WA) by a single research nurse with extensive experience in the care of geriatric patients. Definitions of variables were established and refined during a 2-week preliminary enrollment period during which time the research nurse received training in data abstraction methods. Data collected during the preliminary enrollment period were from patients who presented to the ED prior to June 20 and were not included in analysis. During data abstraction, weekly meetings were held between the research nurse and other study authors to answer questions. Triage scores were obtained from the ED’s computerized patient record system (The T-system EV, version 2.5, 2001–2005; T-System Inc., Dallas, TX).
The determination of whether a patient received an immediate intervention was obtained from a review of the entire nursing and physician record for each patient. An immediate intervention was defined based on criteria established by the ESI version 4 guidelines (Table 1).14 The ESI Handbook does not define a specific time period during which a life-saving intervention must occur to be “immediate.”9 To improve the consistency and reproducibility of our determination of the occurrence of an immediate intervention, we further defined an immediate life-saving intervention as one that occurred in the first hour following patient arrival to the ED. All cases designated as receiving an immediate intervention by the research nurse were reviewed by another study author to confirm that the case met the previously defined criteria and to determine whether an adverse outcome resulted from a delay in care in the ED. This determination was made based on a review of the entire ED and inpatient chart and specifically concerned the nature of the intervention provided, the time until the intervention was provided, and the patient’s ED and hospital course. To identify patients who may have needed an immediate intervention but did not receive one, an additional study author reviewed all patients in the study population who were admitted to the ICU or who died within 24 hours.
|1. Airway and breathing support, including intubation or emergent noninvasive positive pressure ventilation.|
|2. Electrical therapy, including defibrillation, emergent cardioversion, or external pacing.|
|3. Procedures, including chest needle decompression, pericardiocentesis, or open thoracotomy.|
|4. Hemodynamic support, including significant intravenous fluid resuscitation in the setting of hypotension, blood administration, or control of major bleeding.|
|5. Emergency medications, including naloxone, dextrose, atropine, adenosine, epinephrine, or vasopressors.|
A weighted sample of randomly selected patients, 18 of whom received an immediate intervention and 32 of whom did not, was reviewed by one of the study authors (KB) who is an emergency physician. This sample was generated using a random number table to identify 34 patients from the total population and an additional 16 patients from the immediate intervention population. The reviewing physician was provided with the definition of immediate intervention and the entire ED medical record for each patient, but was blinded to the triage ESI score, the proportion of patients with an immediate intervention in the weighted sample, and the categorization by the research nurse as to whether the patient had received an immediate life-saving intervention. For each of the 50 charts, this physician designated the case as receiving or not receiving an immediate intervention.
Additionally, the triage notes for these 50 patients were reviewed by an expert triage nurse blinded to the initial ESI triage level designation. This expert triage nurse (DT) has both taught and studied the ESI and has been involved in numerous revisions of the ESI criteria. The expert triage nurse gave each of these 50 cases an ESI designation, and the initial triage nurse and expert triage nurse designations were compared to assess for triage nurse compliance with ESI triage criteria. Although the triage nurses who made the initial ESI designations and the expert triage nurse had access to the ESI criteria while they were making triage level decisions, the expert triage nurse was encouraged to apply the ESI criteria to each patient. The triage nurses who made the initial designations were not notified that a study was being conducted and were not provided with any instructions.
To allow a comparison of elder adult ESI triage levels to those of nonelder adults, ESI levels for nonelder adults were obtained from the ED’s computerized medical records for the 1-month study period. Further analysis was not performed on the nonelder population.
A sample size determination was not performed for this study. We did not know what the rate of immediate interventions would be in this population. Our objective was to provide an estimate of the sensitivity and specificity of ESI for identifying patients with an immediate intervention, and we felt that data collection over a 1-month period should be sufficient to allow such an estimate. The sensitivity and specificity of ESI Level 1 for identifying patients receiving an immediate intervention were calculated with 95% confidence intervals (CIs). Interrater reliability between the study nurse and the blinded physician as to whether a patient received an immediate intervention was calculated using the kappa statistic. The interrater reliability between triage nurses and the expert triage nurse was determined using the weighted kappa statistic.
During the enrollment period, 4188 adult patients presented to the ED, of whom 782 (18.7%) were 65 years or older. Characteristics of all patients 65 years and over and the subset of patients receiving an immediate intervention were similar, except in regard to disposition (Table 2). Twenty-six (3.3%) of the 782 elders seen in the ED during the study period received an immediate intervention. Nine (1.2%) of the 782 patients were missing triage acuity ratings; none of these nine received an immediate intervention. The most common types of immediate interventions were intubation or emergent noninvasive positive pressure ventilation (15 patients) and emergent medications (six patients). The primary source of triage information was someone other than the patient for 313 of 782 (40%) of all patients 65 years or older and 22 of 26 (85%) of patients receiving an immediate intervention. Prehospital emergency medical services (EMS) providers and family members were the primary source of information for 22 and 14% of all patients, respectively, and 65 and 19% of patient receiving an immediate intervention. Among all patients 65 years and older who presented to the ED during the study period, the admission rate was 58%, and the ICU admission rate was 4%. All 26 patients who received an immediate intervention were admitted, and 12 of these (46%) were admitted to the ICU. Of the 26 patients receiving immediate interventions, eight (31%) died prior to hospital discharge, and two died within the first 24 hours. Thirty-four patients who did not receive an immediate intervention died in the first 24 hours (n = 3) or were admitted to the ICU (n = 31). Based on review of their ED charts, none were felt to have been candidates for an immediate intervention.
|Characteristic||All Patients (n = 782)||Patients Receiving an Immediate Intervention (n = 26)|
|Female||450 (58)||11 (42)|
|Male||332 (42)||15 (58)|
|White||528 (68)||18 (70)|
|Black or African American||219 (28)||3 (12)|
|Other||35 (5)||5 (19)|
|Median age, yr (range)||76 (65–99)||74 (65–92)|
|Age 65–74, yr||342 (44)||14 (54)|
|Age 75–84, yr||286 (37)||7 (27)|
|Age ≥ 85, yr||154 (20)||5 (19)|
|Discharged||421 (54)||14 (54)|
|Admitted—non-ICU||31 (4)||12 (46)|
|Admitted—ICU||3 (0.4)||2 (8)|
|Death in first 24 hours|
Patients 65 years of age or older had a larger proportion of ESI scores of 1, 2, and 3 than did nonelder adults (Table 3). Of the 26 patients receiving an immediate intervention, 11 had an ESI of 1, nine had an ESI of 2, and six had an ESI of 3. The sensitivity of an ESI score of 1 for identifying patients receiving an immediate intervention was 42% (95% CI = 26% to 61%), and the specificity was 99.1% (95% CI = 98.1% to 99.5%; Table 4).
|ESI Level||Patients Aged 18–64 yr (n = 3,364)||Patients ≥ 65 yr (n = 773)||Patients ≥ 65 yr With an Immediate Intervention (n = 26)|
|1||40 (1.2)||18 (2)||11 (42)|
|2||562 (16.7)||176 (23)||9 (35)|
|3||1604 (47.7)||461 (60)||6 (23)|
|4||911 (27.1)||100 (13)||0|
|5||247 (7.3)||18 (2)||0|
|ESI Level||Immediate Intervention|
There were six patients during the study period who received an immediate intervention but received an ESI score of 3. As both ESI recommendations and the practice in our ED is to allow patients with an ESI score of 3 to wait in the waiting room if a bed is not available, these six patients were at particularly high risk for an adverse outcome due to a delay in care. The immediate interventions for these patients were positive pressure ventilation for a patient with respiratory distress due to an exacerbation of chronic obstructive pulmonary disease, large-volume resuscitation for hypotension (two cases), intravenous naloxone for altered mental status of unknown etiology (two cases), and intravenous dextrose for altered mental status and hypoglycemia. As per the study definition, all six patients received the immediate intervention within 1 hour of arrival to triage. None of these six patients died during the subsequent hospitalization. In reviewing the ED course, we did not see evidence for a delay in ED care for these six patients or the remaining 20 patients who received an immediate intervention but had ESI levels of 1 or 2.
For the 50 cases in which an emergency physician reviewed the designation of an immediate intervention, the research nurse and second author agreed on the occurrence of an immediate intervention on 48 cases (κ = 0.91). For one case, the research nurse failed to identify an immediate intervention that was identified by the physician. In the other case, the research nurse identified an immediate intervention that was not felt to be an immediate intervention by the physician. These two discrepancies were reviewed by the study group and both were decided, for the purpose of analysis, to be consistent with the designation by the physician reviewer.
In comparison to the review by the expert triage nurse, triage nurse ESI level designations were the same in 33 cases (weighted κ = 0.61). For the 17 cases where they differed, triage nurses gave higher acuity triage designations than the expert nurse for four patients and lower acuity triage designations for 13 patients. Of the 18 cases in this weighted randomly selected subset of cases in which a patient received an immediate intervention, 13 were designated as ESI Level 1 by the expert triage nurse, nine were designated as ESI Level 1 by the triage nurse, and four cases were designated as ESI > 1 by both the expert and nonexpert triage nurses.
Immediate interventions occurred infrequently in this population of ED patients aged 65 years and older. When they did, more than half of the patients had an ESI score other than 1, indicating a lack of agreement between the ESI Level 1 definition, which should include all patients in need of an immediate intervention, and the application of ESI by our triage nurses. There are several possible reasons for this discrepancy.
Accurate triage of elder patients, in particular the identification of elder patients in need of an immediate intervention, may be more difficult than for nonelder adults for several reasons. High rates of chronic illness in this population may make acute illness more difficult to identify. High rates of triage by proxy, which in this study occurred in 40% of all patients and 85% of patients receiving immediate interventions, may make it difficult for triage nurses and other medical providers to identify the severity and acuity of an elder’s condition. Finally, physiologic and pharmacologic factors may limit changes in vital signs in response to illness or injury,15,16 limiting the ability of triage nurses to rely on vital sign abnormalities to identify life-threatening illness in this population.
Both limitations of existing ESI criteria, and a failure to appropriately apply criteria, may have contributed to the rate of undertriage. The high number of cases in which the triage nurses undertriaged the patient relative to the expert triage nurse (13 of 50 total cases and 5 of 13 cases in which a patient received an immediate intervention) suggests that undertriage in this population may be in part due to a failure of triage nurses to appropriately apply ESI criteria. One possible explanation for the apparent inappropriate application of ESI criteria in this population may be an unconscious bias against elders by triage nurses.17,18 However, there were four cases in which an immediate intervention occurred in which neither the triage nurse nor expert triage nurse review gave the patient an ESI Level 1 designation, suggesting that there may also be a limitation in the ESI criteria for identifying these patients. Both problems with the application of triage criteria to elders and problems with the criteria themselves have been previously described in the context of the prehospital triage of elder trauma patients15,18,19 and in ED triage instruments.17,20 The study was not designed to identify the cause of undertriage in elders, and further study is needed to attempt to better define the extent of this problem, the risk it poses to elder ED patients, and ways to address it.
Our results suggest that additional triage nurse training in the application of ESI criteria, and revisions to the ESI criteria, may serve to improve the triage of elder patients with acute illness and injury. Geriatric-specific triage criteria, including geriatric-specific definitions of abnormal vital signs, may serve to improve the accuracy of ESI in identifying elder patients in need of an immediate intervention. Further study is needed to better define the value of either additional training or changes in ESI criteria. Future studies might also consider the performance of ESI for elders in nonacademic settings, the effect of ED crowding on the accuracy of ESI, and whether certain demographic groups are at increased risk for undertriage.
Balancing sensitivity and specificity of a clinical decision tool requires tradeoffs. To increase the sensitivity of triage criteria for elders in need of an immediate intervention, an increased number of patients not in need of an immediate intervention would be given an ESI Level 1 designation. The impact of these false-positive cases on ED flow and the timely care of all patients must be carefully considered prior to embracing a revision to existing triage criteria. However, the risk of undertriaging a patient and delaying an immediate intervention must be taken seriously.
This study was conducted at a single ED, and the application of ESI by triage nurses in other EDs may differ. Triage nurse training and experience in evaluating and caring for elder patients likely influences the performance of ESI for this population.
In reviewing the charts of those patients who received an immediate intervention but were given an ESI score other than 1, we did not see evidence of either a significant delay in care or an adverse outcome due to a delay in care. Therefore, the argument could be made that the ESI designation did not matter for these patients; they still got what they needed. Although we did not identify such an event in our study, we believe that incorrect ESI designations for patients in need of an immediate intervention place these patients at increased risk for a delay in care and associated morbidity and mortality. This risk is probably particularly high in crowded EDs where patients may have extended waiting room stays if they are undertriaged, or in EDs that are understaffed with physicians or nurses, resulting in long times prior to evaluation. Our data suggest that in our ED, adverse events probably occur in only a small portion of undertriaged patients; a larger study would be needed to identify the rates of adverse events associated with undertriage.
In our study, the sensitivity of an ESI level of 1 for identifying patients in need of an immediate intervention was 42% (95% CI = 26% to 61%). A similar measurement of the performance of ESI is not available for nonelders, and we do not know if the low sensitivity we identified for elder patients reflects a characteristic of the ESI triage instrument as applied to elders or to all patients.
For the purpose of comparison with triage nurse ESI levels, we generated an expert triage nurse ESI level using her review of the chart. The expert triage nurse did not see the patient. This is an imperfect method for comparing triage scores by regular and expert triage nurses, because the normal triage process relies upon both information that is included in the chart and information obtained from seeing and interacting with the patient that is often not included in the chart. Our finding that triage nurses were more likely to undertriage than overtriage elders when compared to the expert nurse may have been influenced by our method of generating expert nurse ESI levels.
Our methods were designed to identify all patients who received an immediate intervention. We did not attempt to determine if the immediate intervention was appropriate or helpful. If we had instead attempted to measure the sensitivity of an ESI level of 1 for patients who benefited from an immediate intervention, the sensitivity of the triage instrument would have been different. It is also possible that a patient required an immediate intervention but did not get one in the first hour. We reviewed all deaths during the first 24 hours and ICU admissions in an attempt to identify such individuals. However, it is possible that one or more elderly patients in need of an immediate intervention presented to the ED during the study period but was not identified by our methods either because no immediate intervention was provided and the patient did not die or get admitted to the ICU or because the study nurse missed a case in which an immediate intervention was provided.
We did not collect information on “do not resuscitate” (DNR) orders on our patients. However, there was at least one patient who was DNR, who would have received an immediate intervention if he or she did not have a DNR order. This patient died within 24 hours of ED arrival. None of the 26 patients who received an immediate intervention in our study had a DNR order documented in the triage or ED note. It may be true that patients with a DNR order with them at triage receive lower triage scores because of this order, and this may or may not be appropriate. Because of variations in the meaning of DNR and in the availability of accurate DNR information at the time of triage, careful consideration would be needed to study the effect of DNR status on the triage process.
There may be characteristics of elder patients, such as age, sex, race, residing in a nursing home, or having a history of dementia, that are associated with undertriage. Our study was not powered to allow for the identification of differences between patients receiving an immediate intervention with an ESI level of 1 and those with an ESI level more than 1. A larger study would be needed to determine if such characteristics are associated with undertriage in elders.
Emergency department overcrowding may affect triage designations. Triage nurses may undertriage when the ED is crowded to avoid having a patient with a high acuity score, such as a 1 or 2, wait in the waiting room before a room becomes available. Undertriage and delays in care for patients in need of an immediate intervention may be a more serious problem in severely overcrowded EDs and at times of day when overcrowding is worse. We did not attempt to assess the effect of ED crowding on ESI designation, but future studies of triage might consider evaluation of the effect of ED crowding on the triage process.
An Emergency Severity Index Level 1 designation is insensitive for identifying elder patients receiving an immediate life-saving intervention. Changes to the index criteria for elders, or further training of triage nurses in the application of the criteria, may be warranted to improve the performance of the Emergency Severity Index in this population.
- 2National Hospital Ambulatory Medical Care Survey: 2003 Emergency Department Summary. Advanced Data from Vital and Health Statistics. National Center for Health Statistics, 2005; 358:1–39., .
- 14Emergency Severity Index, Version 4: Implementation Handbook. AHRQ Publication No. 05-0046-2. Washington, DC: Agency for Healthcare Research and Quality, 2005., , , et al.