To develop a new paediatric illness severity score, called inpatient triage, assessment and treatment (ITAT), for resource-limited settings to identify hospitalised patients at highest risk of death and facilitate urgent clinical re-evaluation.
We performed a nested case–control study at a Malawian referral hospital. The ITAT score was derived from four equally weighted variables, yielding a cumulative score between 0 and 8. Variables included oxygen saturation, temperature, and age-adjusted heart and respiratory rates. We compared the ITAT score between cases (deaths) and controls (discharges) in predicting death within 2 days. Our analysis includes predictive statistics, bivariable and multivariable logistic regression, and calculation of data-driven scores.
A total of 54 cases and 161 controls were included in the analysis. The area under the receiver operating characteristic curve was 0.76. At an ITAT cut-off of 4, the sensitivity, specificity and likelihood ratio were 0.44, 0.86 and 1.70, respectively. A cumulative ITAT score of 4 or higher was associated with increased odds of death (OR 4.80; 95% CI 2.39–9.64). A score of 2 for all individual vital signs was a statistically significant independent predictor of death.
We developed an inpatient triage tool (ITAT) appropriate for resource-constrained hospitals that identifies high-risk children after hospital admission. Further research is needed to study how best to operationalise ITAT in developing countries.
Développer un nouveau score de sévérité de la maladie pédiatrique, appelé Triage des Patients Hospitalisés, Evaluation et Traitement (ITAT), pour les régions à ressources limitées pour l'identification des patients hospitalisés à risque le plus élevé de décès et pour faciliter une réévaluation clinique d'urgence.
Nous avons effectué une étude cas-témoin imbriquée dans un hôpital de référence au Malawi. Le score ITAT a été dérivé de 4 variables équitablement pondérées, ce qui a donné un score cumulatif entre 0 et 8. Les variables comprenaient la saturation en oxygène, la température, les fréquences cardiaques et respiratoires ajustés selon l'âge. Nous avons comparé le score ITAT entre les cas (décès) et les témoins (libérés) pour prédire le décès dans les 2 jours. Notre analyse s'est appuyée sur les statistiques prédictifs, la régression logistique bivariée et multivariée, et le calcul des scores guidé par les données.
Un total de 54 cas et 161 témoins ont été inclus dans l'analyse. L'aire sous la courbe ROC était de 0,76. Pour un seuil ITAT de 4, les rapports de sensibilité, de spécificité et de probabilité étaient de 0,44; 0,86 et 1,70, respectivement. Un score ITAT cumulatif de 4 ou plus était associé à une probabilité accrue de décès (OR: 4,80; IC95%: 2,39 à 9,64). Un score de 2 pour tous les signes vitaux individuels était un facteur prédictif indépendant de décès statistiquement significatif.
Nous avons développé un outil de triage en milieu hospitalier (ITAT) approprié pour les hôpitaux à ressources limitées, qui identifie les enfants à risque élevé après l'hospitalisation. Des recherches supplémentaires sont nécessaires pour étudier la meilleure manière d'appliquer l'ITAT dans les pays en développement.
Desarrollar un nuevo sistema de puntuación para evaluar la gravedad de la enfermedad en pediatría llamado ITAT (por sus siglas en inglés Inpatient Triage, Assessment, and Treatment) para utilizar en lugares con recursos limitados que identifique a los pacientes hospitalizados con un mayor riesgo de muerte y facilite su reevaluación clínica urgente.
Hemos realizado un estudio caso-control anidado en un hospital de referencia de Malawi. El puntaje ITAT se derivó de 4 variables con el mismo peso, con un puntaje acumulativo de entre 0 y 8. Las variables incluían la saturación de oxígeno, la temperatura y las tasas cardiaca y respiratoria ajustadas por edad. Hemos comparado el puntaje ITAT entre casos (muertes) y controles (dados de alta) para predecir la muerte en los 2 días siguientes. Nuestro análisis incluye estadísticas predictivas, regresión logística bivariada y multinomial, y el cálculo de puntuaciones derivadas de datos.
En el análisis se incluyeron un total de 54 casos y 161 controles. El área bajo la curva de la característica operativa del receptor (ROC) era de 0.76. En un punto de corte ITAT de 4, la sensibilidad, especificidad, y tasa de probabilidades eran 0.44, 0.86, y 1.70, respectivamente. Un puntaje ITAT acumulativo de 4 o más estaba asociado con un aumento en la probabilidad de muerte (OR: 4.80; IC 95%: 2.39–9.64). Un puntaje de 2 para todos los signos vitales individuales era estadísticamente significativo, independientemente del vaticinador de muerte.
Hemos desarrollado una herramienta de evaluación de pacientes ingresados (ITAT), apropiada para hospitales con recursos limitados, que identifica a los niños hospitalizados con un mayor riesgo. Es necesario llevar a cabo más estudios para determinar la mejor forma de hacer que el ITAT sea operativo en países en vías de desarrollo.
Similar to many resource-limited settings, Malawi struggles with both shortages and inadequate training of medical staff (Liese & Dussault 2004; WHO 2006; Mueller et al. 2011). In a recent survey, only 42% of expected hospital staff days were adequately filled, just 63% of healthcare staff felt they were adequately trained, and merely 57% felt they were adequately supervised (Mueller et al. 2011). The country has taken steps to alleviate some of these inadequacies by training providers in emergency triage, assessment and treatment (ETAT; Gove et al. 1999), a validated component of the World Health Organization (WHO) integrated management of childhood illness (IMCI) programme that provides simplified algorithms for triaging and treating patients on hospital admission. Although ETAT has led to dramatic improvements in mortality when used at initial triage (Robertson & Molyneux 2001; Molyneux et al. 2006; Robinson et al. 2011), once patients are admitted to the inpatient ward, no further formalised surveillance system exists to provide continued monitoring and timely clinical intervention. As a result, students, nurses and other junior healthcare workers are tasked with identifying deteriorating ward patients in often overcrowded facilities despite limited expertise. A recent government audit in Malawi found that inpatient paediatric mortality is often due to inadequate monitoring and delays in instituting emergency treatment (MMoH 2010). Additional inpatient tools similar to ETAT are needed to help undertrained staff recognise and intervene on deteriorating paediatric ward patients after admission.
In developed countries, severity of illness scores, often referred to as Pediatric Early Warning System (PEWS) scores, have gained popularity as inpatient tools to objectively identify and triage patients prior to their need for urgent resuscitation. Numerous scoring systems have been introduced with varying levels of complexity (Monaghan 2005; Tibballs et al. 2005; Duncan et al. 2006; Haines et al. 2006; Tibballs & Kinney 2006; Duncan 2007; Egdell et al. 2008; Edwards et al. 2009; Parshuram et al. 2009; Tucker et al. 2009), leading to substantial reductions in emergent resuscitation (Brilli et al. 2007), clinical deterioration events (Parshuram et al. 2011) and ward respiratory arrests (Hunt et al. 2008), as well as increased hospital staff confidence (Monaghan 2005; Parshuram et al. 2011) when combined with a rapid response team. although adult severity of illness scores have been successfully implemented in South Africa (Rosedale et al. 2011) and Tanzania (Rylance et al. 2009), no paediatric scoring systems have been reported in similar developing settings.
Building on the successful implementation of the ETAT programme during hospital admission, we aimed to develop a simplified PEWS score that could provide continued monitoring and triage of patients throughout their hospitalisation. Called inpatient triage, assessment and treatment (ITAT), this tool was designed for resource-limited settings in which staff may have limited time or training to adequately identify deteriorating patients prior to death. Once identified, these high-risk patients could then be clinically reassessed with changes in treatment or possible referral to a higher level of supervision.
Kamuzu Central Hospital (KCH) is a large referral hospital in Lilongwe, Malawi. Non-surgical patients are initially triaged on admission, then admitted to one of several wards, with the most critically ill sent to the acute care, malnutrition or high-dependency wards. In 2010, KCH had a patient-to-nurse ratio of around 25 to 50:1 during the day and often exceeding 100:1 at night (Robert Milazi, unpublished data), an inpatient child mortality rate of 7%, and a paediatric HIV prevalence of 8.8% (McCollum et al. 2011). Although 3–4 physicians technically supervise the children's wards, in reality, they often have many other obligations and the majority of clinical decisions are made by junior doctors, medical students and clinical officers (COs). COs are a cadre of non-physician clinicians that provide the majority of clinical care in Malawi after receiving 3 years of theoretical and practical training followed by 1 year of clinical internship.
All children (age <15 years) on the acute care and malnutrition wards at KCH from December 2010 to April 2011 (rainy season) were eligible for the study, which included the majority of children admitted to the hospital. Children on the surgical, high-dependency unit and non-acute wards were given ITAT scoring sheets in their charts but were not included in the study because they were either less ill or were already receiving closely supervised care and the goal of the programme was to identify paediatric patients on the general ward in need of higher levels of care.
Vital signs are collected at KCH using automated vital sign poles, while respiratory rate is determined by observation. The ITAT score is derived from 4 vital signs: heart rate, respiratory rate, oxygen saturation and temperature, with ranges based on a previously validated PEWS score (Duncan et al. 2006). A score from 0 to 2 was given for each vital sign based on the age-associated degree of abnormality, with higher scores resulting from more abnormal vital signs (Table 1). The cumulative ITAT score had a range from 0 to 8. The previously validated individual parameters including heart rate (Haines et al. 2006; Parshuram et al. 2009), respiratory rate (Haines et al. 2006; Parshuram et al. 2009), oxygen saturation (Haines et al. 2006; Parshuram et al. 2009) and temperature (Parshuram et al. 2009) are all objective and important components of the paediatric advanced life support guidelines (Kleinman et al. 2010). Blood pressure was not included as it is technically more difficult, time consuming and is generally considered a late sign of paediatric shock (Ceneviva et al. 1998). An assessment of mental status was also excluded as this was felt to be a more complicated and subjective evaluation. Clinicians were notified to review all patients with ITAT scores ≥4, which we hypothesised would be the optimal threshold score, as it offered a balance of sensitivity in identifying high-risk patients, but also enough specificity that it would not overburden the clinicians in this very resource-limited setting with ‘false positive' notifications.
Table 1. Inpatient triage, assessment and treatment (ITAT) score
Oxygen saturation (%)
The ITAT analysis occurred within a three-phase parent study evaluating two new programmes: (i) the provision of vital sign equipment and the implementation of our ITAT score, and (ii) the introduction of a new cadre of healthcare workers, called ‘Vital Sign Assistants,’ who were to perform routine vital signs including the ITAT score with appropriate clinician notification (Olson et al. 2013).
We used a prospective nested case–control design to retain an appropriate distribution of time in the hospital among controls. Children were eligible for inclusion in the analysis if they had non-missing data for final outcome (death or discharge from ward), date of outcome, date of vital sign collection and at least three of the four ITAT vital signs (oxygen saturation, temperature, heart rate and respiratory rate). In subjects with only three vital signs, the missing vital sign was imputed using single imputation [‘proc mi’ procedure in SAS (SAS Institute, Cary, NC, USA)] to obtain maximum likelihood estimates of the mean vector and covariance matrix. The mean, standard deviation, minimum, maximum and covariance matrix of all vital sign measures remained similar before and after the imputation. Descriptive statistics, including demographic data and diagnoses, were reviewed using SAS (SAS Institute).
Children who died on the acute care or malnutrition wards during the study period were considered cases. Each case's last set of vital signs taken before death was used. If the last set of vital signs was taken more than 2 days before death, the case was excluded. For each case, three controls were selected among a randomly sorted list of non-case vital sign sets, yielding a 1:3 case to control ratio. Sample size was limited to the number of subjects enroled in the parent study during the same time period. Only vital sign sets that fell within 2 days of the case's death were used as controls. As is typical and appropriate in nested case–control studies, controls could become a case at a later point in time; a control could serve as a control more than once, as sets of vital sign collections were the unit of analysis (Rothman et al. 2008). However, the same set of vital signs for an individual patient was not used more than once. Cumulative ITAT scores (ITAT score of 0–8) were calculated based on individual vital sign values, and used for analysis.
ITAT score development
To determine predictors of mortality, ‘high’ scores, defined as having a cumulative ITAT score equal to or greater than the pre-determined cut-off, were compared to ‘low’ scores, defined as having a cumulative ITAT score lower than the pre-determined cut-off, for all possible cumulative scores (0–8). Sensitivity (probability of having a high cumulative ITAT score among children that died), specificity (probability of having a low cumulative ITAT score among children that did not die) and stratum-specific likelihood ratio (LR) were calculated for each ITAT score cut-off. Bayes Theorem was used to calculate positive predictive value (PPV) and negative predictive value (NPV) for a cut-off score of 4, using the prevalence of death in our study population. Receiver operating characteristic (ROC) curves were then plotted using the sensitivity and specificity (1 − specificity) for each ITAT score cut-off, and the area under the ROC curve was calculated using the trapezoidal rule. Microsoft Excel 2010 (Microsoft, Redmond, WA, USA) was used for the above calculations.
Bivariable and multivariable logistic regression with robust variance was used to estimate odds ratios and 95% confidence intervals for the association between each predictor variable (oxygen saturation, temperature, heart rate and respiratory rate) and final outcome (death or discharged from ward). Odds ratios approximated rate ratios due to density sampling. Predictor variables were assessed in the following ways: continuous form using various transformations, including splines; categories using disjoint indicator variables; and by their given score. We used disjoint indicator variables based on the Akaike Information Criterion (AIC). Variables were assessed for collinearity and extreme observations. All modelling was carried out using SAS version 9.2 (SAS Institute).
Data-driven ITAT scores were estimated to evaluate the utility of the assigned ITAT scores. Beta coefficients, which provide a natural linear interpretation for classification of the odds of disease, were derived from each of the predictor variables in the multivariable logistic regression model and were used in two ways: (i) as their unrounded value; and (ii) rounded to the nearest integer to obtain the data-driven ITAT score. The individual ITAT scores were then summed to obtain the cumulative data-driven ITAT score. Sensitivity, specificity, stratum-specific LR and ROC curves were repeated using the data-driven ITAT scores.
The study was approved by the University of North Carolina Internal Review Board and the Malawi National Health Sciences Research Committee. As vital signs are a routine component of patient care, individual patient consent was not required, although information pamphlets regarding the study were still made available throughout the children's wards.
In total, 1615 children were eligible for study inclusion, while 377 were excluded for insufficient vital signs or outcome status, and 1084 had no vital signs taken during their hospitalisation (Table 2). Imputation was used on 630 of the 3983 eligible sets of vital signs with 1 missing vital sign. The remaining vital sign sets included all four vital signs. Missing data were due to incomplete documentation in patient charts. Fifty-four cases met eligibility criteria; with 161 controls included in the analysis (1 case only had 2 controls instead of 3 because only 2 controls met eligibility criteria). Case and control patients were unevenly distributed within the parent study, with the majority (41) of cases (and therefore controls) in the final third of the rainy (malaria) season. For both cases and controls, malaria, diarrhoea/dehydration and pneumonia were the top three diagnoses.
Nine children not represented who died but had vital signs collected >2 days before death, so eligible for control group only.
0 to 1 months
2 to 6 months
7 to <12 months
1 to 2 years
3 to 5 years
Day of week admitted
Table 3 shows the sensitivity, specificity and stratum-specific LR of each cumulative ITAT score in predicting mortality within 2 days. At a score of 4, which was the pre-determined threshold in which clinicians were notified to evaluate a patient, the sensitivity, specificity and LR were 0.44, 0.86 and 1.70, respectively. The area under the ROC curve was 0.76 (Figure 1). The PPV and NPV for a cut-off of 4 was 0.18 and 0.96, respectively, using the prevalence of death in our study population (6.7%, 206/3066).
Table 3. Inpatient triage, assessment and treatment score sensitivity, specificity, and likelihood ratios (LR)
Individual and composite predictors of mortality are shown in Table 4. Having a cumulative ITAT score of 4 or higher was independently associated with an increased odds of death, compared to having an ITAT score <4 (crude OR 4.80, 95% CI 2.39–9.64). For all vital signs except respiratory rate, a score of 1 (compared to a score of 0) did not predict death. A score of 2 (compared to a score of 0) was a significant independent predictor of death for all vital signs. Beta and data-driven ITAT scores based on multivariable regression are also included but did not suggest that any individual vital sign should have greater weight in determining the ITAT score.
Table 4. Predictors of mortality, using disjoint indicator coding for scores
Individual vital signs
All vital signs
Odds ratio (95% CI)
Odds ratio (95% CI)
SE, Standard Error.
Overall risk score
≥4 vs. <4
1 vs. 0
2 vs. 0
1 vs. 0
2 vs. 0
1 vs. 0
2 vs. 0
1 vs. 0
2 vs. 0
The sensitivity, specificity and LR for data-driven ITAT scores are described in Table 5 and again show little improvement compared to the original ITAT score. The data-driven area under the ROC curve was 0.78.
Table 5. Data-driven score sensitivity, specificity, and likelihood ratios (LR)
In clinical settings, lacking sufficient personnel and training such as KCH, improved inpatient monitoring and triage systems are necessary to identify and treat critically ill paediatric patients prior to deterioration and death. The ITAT scoring system includes a simple objective algorithm that builds on an already existing programme to continue monitoring and triage after a patient has been admitted to the ward. Using this scoring system, healthcare personnel can quickly and efficiently identify children at high risk of death on the ward, allowing clinicians the opportunity to review and intervene on those patients appropriately.
Previously described severity of illness scores have gained widespread acceptance in the developed world, showing improved outcomes following their implementation (Brilli et al. 2007; Hunt et al. 2008; Parshuram et al. 2011). Several limitations, however, inhibit their potential effectiveness in resource-limited settings. One such limitation is that greater performance comes with the cost of greater complexity. The best performing scores such as those developed by Duncan et al. (2006; sensitivity 78%, specificity 95%) and Parshuram et al. (2009; 82% sensitivity, 93% specificity), include many items (up to 16) or are time intensive and clinically subjective (level of consciousness, respiratory effort, airway threat). In addition, several scoring systems include items that are dependent on treatment already provided such as oxygen (Monaghan 2005; Duncan et al. 2006; Egdell et al. 2008; Edwards et al. 2009; Parshuram et al. 2009) intravenous fluids (Duncan et al. 2006; Haines et al. 2006) and nebulised adrenaline (Haines et al. 2006). These items have less utility in settings where their associated treatments are underutilised or not available. Previously reported scoring systems are designed to identify acutely deteriorating patients, often within hours of their arrest or transfer to intensive care. In settings lacking resources for rapid response teams, intensive care, or even routine vital signs, scoring systems may be more appropriately used to guide overall management than to provide triggers for urgent resuscitation.
The greatest strengths of the ITAT score are that it can be carried out quickly and that it requires only a minimal amount of clinical expertise. With only four easily measured items, all of which can be objectively measured using vital sign equipment or direct observation, a single ITAT score assessment can be completed more quickly than previously reported scores. Furthermore, the ITAT score does not include items requiring high levels of clinical knowledge such as evaluations of mental status and respiratory effort, or items that are based on responses to clinical interventions such as IV boluses and oxygen. The ITAT score shows similar characteristics compared to the data-driven ITAT score, suggesting our algorithm would not benefit from more complicated weight adjustment. This overall simplicity allows for the possibility of task shifting this tool to less costly, capable workers who would require less training, allowing personnel with greater experience to focus on more specialised tasks. The ITAT score may be used as a continuous surveillance system with multiple assessments performed each day, allowing clinicians an objective measure of a patient's status over time and opportunities to review and change management plans where appropriate. In settings with high staff and patient turnover, this tool provides some level of continuity between multiple assessments from different clinicians. The threshold score for clinician notification can also be adjusted depending on human resource constraints. Using our data, decreasing the ITAT threshold score to 3 or greater would more than double the number of clinician notifications (unpublished data).
The ITAT score's simplified design, however, also leads to several limitations. One such limitation is that it lacks the accuracy of more complicated scoring systems. Previously validated scores report an area under the ROC curve of 0.86–0.91 (Duncan et al. 2006; Egdell et al. 2008; Edwards et al. 2009; Tucker et al. 2009; Parshuram et al. 2011) compared with the ITAT score ROC of 0.76, suggesting the more complex scoring systems are better able to discriminate patients with poor outcomes. Similarly, sensitivity and specificity are also slightly decreased with our scoring system. Our use of imputation as well as the use of clinician interventions for ITAT scores ≥4 may have decreased our area under the ROC curve, sensitivity and specificity. The use of single imputation may also slightly overestimate precision. Unlike previous scores that aim to recognise deteriorating patients requiring intervention within hours, the ITAT score is designed to predict paediatric death within 2 days. Thus, while children with an elevated score should be urgently evaluated, it should be emphasised that the score is designed as a surveillance tool on the general ward to provide continuous patient monitoring throughout their hospitalisation. As is common at KCH, a substantial amount of missing data resulted from either incomplete or no vital signs being collected by healthcare staff, which may have introduced selection bias into our study. The absence of vital signs was not always at random. Children that died were more likely to have zero vital sign assessments, which was expected as the most critical children were more likely to die immediately following admission to the ward, prior to any vital sign collection. These children may have received vital signs on initial triage in the ‘under five’ clinic prior to admission, although we chose not to include those vital signs in our analysis as they are often incomplete, poorly documented and would not add value in evaluating our inpatient triage tool. Missing vital sign data were also associated with time of day the outcome occurred, with afternoon or night having higher odds of being missing compared with morning (data not shown). The association between ‘time of day’ and missing vital sign data was likely due to fewer on duty staff at night. Influential observations, or single observations that have a significant impact on the regression function, were numerous in our study, which may be indicative of our relatively small sample size. It is possible that missing data and/or influential observations biased our effect estimates and standard errors. A prospective cohort design was not used in this study because while individuals who died most often did so within 48 h of admission, individuals who survived most often had multiple vital sign sets, thus disproportionately affecting score performance. A case–control design avoided this concern.
In conclusion, we have designed and implemented a simplified paediatric severity of illness score (ITAT) capable of recognising children at high risk of death. Once identified as high risk, patients can be triaged for clinician re-evaluation, possible changes in management or increased levels of supervision. Future studies should investigate the best way to operationalise the ITAT programme in similar settings, including significant changes in the ITAT score as trigger for clinician review, patient outcome measures such as mortality and the evaluation of task shifting the ITAT score to health workers with less training.
We would like to thank our assistant program coordinator, Jean Nkhoma and our Vital Sign Assistants, for their valuable contributions to the success of this programme. We would also like to acknowledge Christopher Wiesen and the UNC Odum Institute for help with data management. This work was supported by a grant from Health Empowering Humanity, the National Institutes of Health, as well as the National Institutes of Health Office of the Director, Fogarty International Center, Office of AIDS Research, National Cancer Center, National Eye Institute, National Heart, Blood, and Lung Institute, National Institute of Dental and Craniofacial Research, National Institute on Drug Abuse, National Institute of Mental Health, National Institute of Allergy and Infectious Diseases, and National Institutes of Health Office of Women's Health and Research through the Fogarty International Clinical Research Scholars and Fellows Program at Vanderbilt University (R24 TW007988) and the American Relief and Recovery Act.