The authors have no relevant financial information or potential conflicts of interest to disclose.
Predicting Hospital Admissions at Emergency Department Triage Using Routine Administrative Data
Version of Record online: 15 AUG 2011
© 2011 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 18, Issue 8, pages 844–850, August 2011
How to Cite
Sun, Y., Heng, B. H., Tay, S. Y. and Seow, E. (2011), Predicting Hospital Admissions at Emergency Department Triage Using Routine Administrative Data. Academic Emergency Medicine, 18: 844–850. doi: 10.1111/j.1553-2712.2011.01125.x
Supervising Editor: Rochelle Fu, PhD.
- Issue online: 15 AUG 2011
- Version of Record online: 15 AUG 2011
- Received October 12, 2010; revisions received January 19, February 24, and March 23, 2011; accepted March 25, 3011.
ACADEMIC EMERGENCY MEDICINE 2011; 18:844–850 © 2011 by the Society for Academic Emergency Medicine
Objectives: To be able to predict, at the time of triage, whether a need for hospital admission exists for emergency department (ED) patients may constitute useful information that could contribute to systemwide hospital changes designed to improve ED throughput. The objective of this study was to develop and validate a predictive model to assess whether a patient is likely to require inpatient admission at the time of ED triage, using routine hospital administrative data.
Methods: Data collected at the time of triage by nurses from patients who visited the ED in 2007 and 2008 were extracted from hospital administrative databases. Variables included were demographics (age, sex, and ethnic group), ED visit or hospital admission in the preceding 3 months, arrival mode, patient acuity category (PAC) of the ED visit, and coexisting chronic diseases (diabetes, hypertension, and dyslipidemia). Chi-square tests were used to study the association between the selected possible risk factors and the need for hospital admission. Logistic regression was applied to develop the prediction model. Data were split for derivation (60%) and validation (40%). Receiver operating characteristic curves and goodness-of-fit tests were applied to the validation data set to evaluate the model.
Results: Of 317,581 ED patient visits, 30.2% resulted in immediate hospital admission. In the developed predictive model, age, PAC status, and arrival mode were most predictive of the need for immediate hospital inpatient admission. The c-statistic of the receiver operating characteristic (ROC) curve was 0.849 (95% confidence interval [CI] = 0.847 to 0.851). The goodness-of-fit test showed that the predicted patients’ admission risks fit the patients’ actual admission status well.
Conclusions: A model for predicting the risk of immediate hospital admission at triage for all-cause ED patients was developed and validated using routinely collected hospital data. Early prediction of the need for hospital admission at the time of triage may help identify patients deserving of early admission planning and resource allocation and thus potentially reduce ED overcrowding.
Overcrowding at emergency departments (EDs) continues to be a problem faced by hospitals in Singapore and in developed countries.1–10 This has contributed to pressures on national health care systems, funders, and providers of ED services. The reasons for overcrowding may be attributed to heavy workload, shortage of inpatient beds, and the long waiting time for admission from the ED to inpatient wards.6,11–14
In the current practice in an ED in Singapore, patients are immediately triaged on arrival by the nurses to different acuity categories. They then wait to see doctors, and the median wait time is about 2 hours.15 After consultation, doctors make the decision to admit or to discharge the patients. If a decision is made to admit a patient, a bed request is made by the doctor to the bed management unit, which will allocate a bed and make necessary preparation and resource allocation. The median waiting time from a bed request to the actual admission is about 4 hours.15
Early prediction of hospital admission among ED patients at triage may help reduce the waiting time at the ED by triggering the admission processes earlier.16 Patients who need admission would thus have a reduced total wait time, as the wait time for a bed would be incorporated into the consultation wait time. By implementing such a prediction model to help make an early assessment of a patient’s admission needs at the time of triage, the efficiency of ED care and patient flow from ED to inpatient wards can be improved.
Many decision support models have been developed and applied in EDs to predict hospital admissions.16–24 However, most of them focus primarily on patient subgroups20,21 or specific diseases.22–24 Very few have taken the view of all-cause admissions.16–19 There is no available model developed in Singapore.
This study aimed to develop a model to predict the need for immediate hospital admission that is likely to follow an episode of ED care, using routinely collected administrative hospital data at the time of triage. Such a model could alert clinicians and relevant hospital system personnel that the likelihood of an individual patient’s need for admission is high, to trigger prompt initiation of the admission processes.
This was a retrospective study using routinely collected hospital data to develop a predictive model. The study was approved by the hospital’s research ethics committee.
Study Setting and Population
The study was conducted in the ED of a 1000-bed tertiary care hospital in Singapore. The ED has approximately 160,000 patient visits per year, about 30% of which result in an admission for inpatient care. We evaluated the use of routinely collected data at triage to identify patients at high risk of hospital admission through the ED. All patient visits at the ED from January 2007 to December 2008 were included.
Data were extracted from the ED clinical information system, “EDWeb,” which stores patients’ demographics, patient acuity category (PAC), diagnoses, utilization, payment, and disposition status of ED visit. Patients who died in the ED or who were directly admitted to an intensive care unit or high dependency unit were excluded. For PAC status, the Ministry of Health of Singapore25 defines patients needing resuscitation, having cardiovascular collapse or in imminent danger of collapse, or who need to be attended to without a moment’s delay as PAC 1; patients not needing resuscitation, having a major emergency, who are ill and nonambulatory, or having severe symptoms as PAC 2; and patients with a minor emergency or who are ambulatory with mild to moderate symptoms as PAC 3. The primary outcome of the study was the admission to the general ward.
Analysis of variance tests and contingency table analysis with chi-square statistics were used to study the crude association between predictors and admission.
For model development, first a random intercept logistic regression model was built to address the effect of intrapatient correlation, as some patients had multiple visits to the ED. The variance of the random effect () output from the SAS proc GLIMMIX (with logit link) was 0.103 (standard error [SE] ± 0.011). The pseudo intraclass correlation (ICC) coefficient was then estimated based on the Snijders-Bosker formula26 as: . Furthermore, the variance inflation factor (VIF), which indicates the relative increase in the variance caused by the ICC, was calculated based on the formula27 as:
where m is the average number of visits per patient.
The small ICC and VIF suggested that the random effects model may not be necessary. Therefore, a regular stepwise logistic regression was applied to identify the significant predictors and to build the prediction model. Candidate predictor variables considered for possible inclusion in the model a priori included in the regression were demographics (age, sex, and ethnic group), chronic conditions (diabetes, hypertension, and dyslipidemia), arrival mode (ambulance, walk-in), PAC status, hospital admission, and ED visit in preceding 3 months.
All potential predictors associated with the need for admission were entered into the model as independent variables, and inpatient admission was the dependent variable. The important predictors to be included in the final model have to be clinically meaningful and be significant at the p < 0.001 level considering the very large sample size of the study. Split validation approach was used, where all patient visits were randomly divided into two data sets: 60% for the derivation set and the remaining 40% for the validation set. The derivation data set was used to develop the model, and the validation data set was subsequently used to validate the model. C-statistics of the receiver operating characteristic (ROC) plot and the Hosmer and Lemeshow goodness-of-fit test were applied on the validation data set, respectively, to assess the discrimination power and the goodness of fit of the model.
Patients from the validation set who visited the ED were then assigned predicted probabilities (risk scores) of being immediately admitted based on the developed prediction model. The risk score with a range of 0 to 1 was stratified into 10 risk categories. The goodness of fit of the model was assessed by the Hosmer and Lemeshow test by comparing the observed actual admissions and predicted risk scores in all 10 risk categories. A risk score of 0.7 was selected as the cutoff: patients who have a predicted risk score of 0.7 and above were considered high-risk patients who need immediate hospital admission. All of the statistical analyses were conducted using SPSS version 18 (SPSS Inc., Chicago, IL), aside from the multilevel modeling, which was conducted using SAS 9.2 (SAS Institute Inc., Cary, NC).
From January 2007 to December 2008, a total of 317,581 ED visits were made by 207,069 patients who were alive at discharge from the ED. Of these, 30.2% resulted in an inpatient admission. Table 1 summarizes the admission rate among ED patients by various characteristics. The admission rate increased progressively with age in a linear relationship. The maximum age-specific rate of admission was 76.1%, among persons aged 85 years and above. The mean (± standard deviation [SD]) age of the admitted patients was 60.1 (SD ±19.9) years, compared to 39.4 (SD ±18.3) years among those not admitted. Significantly higher admission rates were noted for females compared to males; Chinese ethnic group compared to other ethnic groups; those brought in by ambulance compared to walk-ins; those who had prior ED visits or hospital admissions in the preceding 3 months compared to those who did not; those triaged as being PAC 1 or PAC 2 compared to those triaged as PAC 3; and those having diabetes, hypertension, or dyslipidemia compared to those without any of these three chronic conditions.
|Characteristics||Total ED Visits||Visits Resulting in Admission||p-value|
|Age group, yr|
|ED visit in preceding 3 months|
|Admission in preceding 3 months|
|Diabetes, hypertension, dyslipidemia||43,814||27,977||63.9|
In the stepwise regression analysis, all predictors other than sex were significant predictors for hospital admission (all p-values were less than 0.001). Based on the changes in the log likelihood, PAC status, age group, and arrival mode were ranked as the three variables most predictive of the need for immediate hospital admission. The odds of being admitted in PAC 1 patients were 20 times higher than in PAC 3 patients. The odds of being admitted in patients aged 65 years and above were 2.5 to 5.3 times higher than in the 25- to 34-year age group (Table 2). The odds of being admitted for patients who arrived at the ED by ambulance were 1.7 times higher than in patients who walked in. Patients who visited the ED or who were admitted in the preceding three months were more likely to be admitted. Indian and Chinese patients were more likely to be admitted than Malay patients. That the ethnic group was significantly associated with admission may be due to the correlations between the ethnic group and the unmeasured social-economic, behavioral factors, diet habits, lifestyle, etc. The admission risk in patients with diabetes, hypertension, or dyslipidemia was higher than that in patients without these chronic conditions. The more chronic conditions a patient had, the more likely the patient was to be admitted. The developed model was deployed using an equation (see Appendix), in which p is the risk of admission to be predicted, and Y is the logit(p).
|Predictors||Odds Ratio||95% CI|
|Age group, yr|
|ED visit in preceding 3 months|
|Hospital admission in preceding 3 months|
|Diabetes with hypertension||2.7||2.4||3.0|
|Diabetes with hypertension and dyslipidemia||2.6||2.5||2.7|
|Diabetes with dyslipidemia||2.1||1.9||2.2|
|Dyslipidemia with hypertension||1.9||1.8||2.0|
Applying the developed model to the validation data set, and comparing the predicted risk of admission with the actual admission status, the model demonstrated good discriminative power with the c-statistic of the ROC curve being 0.849 (95% confidence interval [CI] = 0.847–0.851), which was almost as same as the c-statistic of the derivation data set 0.850 (95% CI = 0.848–0.852). The model provided a good fit to the data based on the Hosmer-Lemeshow test (p > 0.05). The actual admission rate in risk category 1 was 5.4%, while that in risk category 10 was 95.2%. The higher the predicted risk category, the higher the actual admission rate (Table 3).
|Risk Category||Risk||Total Visits||Admissions||Admitted/Total Visits (%)|
The risk score of 0.7 was applied as cutoff to achieve a high specificity. Patients with risk scores of 0.7 and above were predicted to be admitted. The specificity of the model was 96.8% (95% CI = 96.8%–97.0%), the sensitivity was 33.4% (95% CI = 32.9%–33.8%), and the positive predictive value (PPV) and the negative predictive value (NPV) of the model were 81.6% (95% CI = 81.0%–82.2%) and 71.8% (95% CI = 71.5%–72.1%), respectively.
Among patients who were predicted to be at high risk for admission but were not admitted, more than 25% revisited the ED in 1 month, 48% revisited the ED in 6 months, 4.7% were admitted in 1 month, and 9.6% were admitted in 6 months (Table 4). The higher the risk score, the higher the risk of revisiting the ED or being admitted within 1 to 6 months.
|Risk Category||Risk||Total Visits||ED Revisited (%)||Hospital Admission (%)|
In this study, we have evaluated the use of routine ED data collected at the time of triage to predict immediate hospital admission among general ED patients. Based on the good discrimination power and the nonsignificant goodness-of-fit test of the model on the validation data set, the developed model could be a good decision support tool for clinicians in the ED. Our developed model compares well with other prediction models for general ED patients using routine ED data.16–21 In a model represented by a Bayesian network, the c-statistic of the ROC curve was 0.833.16 Two models developed for patients being transported by the emergency medical services system reported a sensitivity of 62% and a PPV of 59%.17,18 The predictors of the model were actually pre-ED variables, which may not be routinely collected in other countries. Ruger et al.19 have studied how the five-point Canadian Triage Assessment Scale, together with a few clinical parameters, can be useful for identifying admission; however, theirs was an exploratory study, and no prediction model was really developed. A model developed by LaMantia et al.20 for elder (75 years and older) ED patients used predictors of age, ESI level, heart rate, diastolic blood pressure, and chief complaint. The reported c-statistic of the ROC curve was 0.73. Leegon et al.21 developed a prediction model for patients presenting at a pediatric ED, which was represented by neural networks with nine predictors. The c-statistic of the ROC curve of the model was about 0.90. Other statistics of these models were not reported in these papers.
The criterion standard in our study was the patient’s actual admission status. However, patients may be over- or undertreated in the ED. We are unable to assess if the patients are appropriately treated in the current practice without a specifically designed primary study. However, patients who subsequently needed ED reattendances or inpatient admission shortly after an ED discharge could have been undertreated. Among patients who were predicted to need an admission but were not admitted, about 25% revisited the ED, and 5.4% were admitted within 1 month. Targeted programs are needed to assess the true admission needs of these patients and to prevent revisits or subsequent hospital admissions.
The developed model is represented by an equation with the important predictors identified and their corresponding coefficients estimated by logistic regression. The model supports its integration with the existing hospital information system for real-time admission prediction at triage, as all predictors are routinely collected at triage and the model represented by the logistic regression equation can be easily implemented. When a patient presents to the ED, his or her condition will be assessed and triaged by a nurse, and a risk score will be automatically calculated according to the equation. Patients with risk scores above the cutoff will be deemed as high admission risk. The system will alert the triage nurse to trigger a bed request to initiate admission processes earlier. The patient will still wait for the doctors’ consultation as in current practice. The doctor’s decision on whether to admit or not based on a clinical judgment will be the final decision.
How to select the cutoff is usually a tradeoff between the sensitivity and specificity, which is dependent on how the model is used. In this instance, the model is mainly used for resource planning and improving the ED throughput. The ED wants to inform the bed management unit to initiate the admission process early without affecting its operations. Therefore a cutoff of 0.7 was used to predict true admission needs while controlling the false alarms. If the model is used for clinical decision support, then a higher sensitivity is preferred, and 0.3 may be used as the cutoff. In that case, the sensitivity of the model is 76.9%, the specificity of the model is 77.6%, and the PPV and NPV of the model are 59.7 and 88.5%, respectively.
As only routine ED data collected at the time of triage were used for developing the model, the limitation is that only available predictors were used. There may be other important factors missed in the model, like the presenting symptoms, the vital signs, and functional or socioeconomic status. This may limit the discrimination and validation power of the model. Second, the Singapore PAC is a very important predictor of the model locally. PAC is judged by the nurse, based on the patients’ presenting symptoms and vital signs at the time of triage. A model using the patients’ presenting symptoms and vital signs instead of PAC status can make the model more generalizable and may improve the model performance. At the moment, the information on the patients’ presenting symptoms and vital signs were stored in electronic case notes in the format of free text. In our future work, we plan to extract this information into a structured data using text mining tools and to compare the model developed from the symptoms and vital signs to the presented model.
We developed a model represented by an equation for predicting hospital admission at the point of triage for general ED patients. The predictive model, which achieved good validation and calibration performance on the validation data set, helps the triage nurses make an early assessment of a patient’s admission needs and identify patients deserving early admission planning and resource allocation, thus potentially reducing ED overcrowding.
- 1Ministry of Health Information Paper: Waiting Times at Public Sector Emergency Departments. Health Information Management Branch, Ministry of Health. Singapore. Available at: http://www.moh.gov.sg/mohcorp/uploadedFiles/Publications/Information_Papers/2005/07_Waiting_Times_at_Public_Sector_Emergency_Departments.pdf. Accessed May 21, 2011..
- 15Singapore Ministry of Health. Waiting Times in Emergency Department. Singapore Ministry of Health: Healthcare Institution Statistics Report, 2010.
- 16Predicting hospital admission at triage in emergency department patients. AMIA Annu Symp Proc. 2007; 11:937., , .
- 21Predicting hospital admission in a pediatric emergency department using an artificial neural network. AMIA Annu Symp Proc. 2006:1004., , , .
- 25Society for Emergency Medicine in Singapore. Singapore Emergency Medicine Services Patient Acuity Category (PAC) Scale. Available at: http://www.semsonline.org/guidelines/pac.html. Accessed May 12, 2011.
- 26Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. London, England: SAGE Publications, 1999., .
- 27Design and analysis of cluster randomization trials in health research. London, England: Oxford University Press, 2000., .