- Top of page
Similar to many resource-limited settings, Malawi struggles with both shortages and inadequate training of medical staff (Liese & Dussault 2004; WHO 2006; Mueller et al. 2011). In a recent survey, only 42% of expected hospital staff days were adequately filled, just 63% of healthcare staff felt they were adequately trained, and merely 57% felt they were adequately supervised (Mueller et al. 2011). The country has taken steps to alleviate some of these inadequacies by training providers in emergency triage, assessment and treatment (ETAT; Gove et al. 1999), a validated component of the World Health Organization (WHO) integrated management of childhood illness (IMCI) programme that provides simplified algorithms for triaging and treating patients on hospital admission. Although ETAT has led to dramatic improvements in mortality when used at initial triage (Robertson & Molyneux 2001; Molyneux et al. 2006; Robinson et al. 2011), once patients are admitted to the inpatient ward, no further formalised surveillance system exists to provide continued monitoring and timely clinical intervention. As a result, students, nurses and other junior healthcare workers are tasked with identifying deteriorating ward patients in often overcrowded facilities despite limited expertise. A recent government audit in Malawi found that inpatient paediatric mortality is often due to inadequate monitoring and delays in instituting emergency treatment (MMoH 2010). Additional inpatient tools similar to ETAT are needed to help undertrained staff recognise and intervene on deteriorating paediatric ward patients after admission.
In developed countries, severity of illness scores, often referred to as Pediatric Early Warning System (PEWS) scores, have gained popularity as inpatient tools to objectively identify and triage patients prior to their need for urgent resuscitation. Numerous scoring systems have been introduced with varying levels of complexity (Monaghan 2005; Tibballs et al. 2005; Duncan et al. 2006; Haines et al. 2006; Tibballs & Kinney 2006; Duncan 2007; Egdell et al. 2008; Edwards et al. 2009; Parshuram et al. 2009; Tucker et al. 2009), leading to substantial reductions in emergent resuscitation (Brilli et al. 2007), clinical deterioration events (Parshuram et al. 2011) and ward respiratory arrests (Hunt et al. 2008), as well as increased hospital staff confidence (Monaghan 2005; Parshuram et al. 2011) when combined with a rapid response team. although adult severity of illness scores have been successfully implemented in South Africa (Rosedale et al. 2011) and Tanzania (Rylance et al. 2009), no paediatric scoring systems have been reported in similar developing settings.
Building on the successful implementation of the ETAT programme during hospital admission, we aimed to develop a simplified PEWS score that could provide continued monitoring and triage of patients throughout their hospitalisation. Called inpatient triage, assessment and treatment (ITAT), this tool was designed for resource-limited settings in which staff may have limited time or training to adequately identify deteriorating patients prior to death. Once identified, these high-risk patients could then be clinically reassessed with changes in treatment or possible referral to a higher level of supervision.
- Top of page
In total, 1615 children were eligible for study inclusion, while 377 were excluded for insufficient vital signs or outcome status, and 1084 had no vital signs taken during their hospitalisation (Table 2). Imputation was used on 630 of the 3983 eligible sets of vital signs with 1 missing vital sign. The remaining vital sign sets included all four vital signs. Missing data were due to incomplete documentation in patient charts. Fifty-four cases met eligibility criteria; with 161 controls included in the analysis (1 case only had 2 controls instead of 3 because only 2 controls met eligibility criteria). Case and control patients were unevenly distributed within the parent study, with the majority (41) of cases (and therefore controls) in the final third of the rainy (malaria) season. For both cases and controls, malaria, diarrhoea/dehydration and pneumonia were the top three diagnoses.
Table 2. Characteristics of study cohort
| ||Died (N = 54)a n (%)||Discharged (N = 1552) n (%)|
|Male||21 (39.62)||792 (51.13)|
|Female||32 (60.38)||757 (48.87)|
|0 to 1 months||2 (3.70)||102 (6.57)|
|2 to 6 months||5 (9.26)||159 (10.24)|
|7 to <12 months||11 (20.37)||179 (11.53)|
|1 to 2 years||17 (31.48)||532 (34.28)|
|3 to 5 years||10 (18.52)||351 (22.62)|
|6+ years||9 (16.67)||229 (14.76)|
|Admission||51 (94.44)||1522 (98.07)|
|Malnutrition||3 (5.56)||30 (1.93)|
|HIV exposed||3 (15.00)||56 (7.73)|
|HIV infected||4 (20.00)||57 (7.87)|
|HIV uninfected||13 (65.00)||611 (84.39)|
|Convulsions||0 (0.00)||26 (1.68)|
|Diarrhoea/Dehydration||6 (11.11)||120 (7.75)|
|Malaria||37 (68.52)||1117 (72.11)|
|Malnutrition||2 (3.70)||25 (1.61)|
|Pneumonia||5 (9.26)||184 (11.88)|
|Sepsis||1 (1.85)||50 (3.23)|
|Other||3 (5.56)||27 (1.74)|
|Day of week admitted|
|Monday–Friday||50 (92.59)||1189 (76.61)|
|Saturday–Sunday||4 (7.41)||363 (23.39)|
Table 3 shows the sensitivity, specificity and stratum-specific LR of each cumulative ITAT score in predicting mortality within 2 days. At a score of 4, which was the pre-determined threshold in which clinicians were notified to evaluate a patient, the sensitivity, specificity and LR were 0.44, 0.86 and 1.70, respectively. The area under the ROC curve was 0.76 (Figure 1). The PPV and NPV for a cut-off of 4 was 0.18 and 0.96, respectively, using the prevalence of death in our study population (6.7%, 206/3066).
Table 3. Inpatient triage, assessment and treatment score sensitivity, specificity, and likelihood ratios (LR)
Figure 1. The receiver operating characteristics (ROC) curve for the inpatient triage, assessment, and treatment (ITAT) score. For an ITAT score of ≥4, the sensitivity, specificity, and likelihood ratios were 0.44, 0.86, and 1.70, respectively.
Download figure to PowerPoint
Individual and composite predictors of mortality are shown in Table 4. Having a cumulative ITAT score of 4 or higher was independently associated with an increased odds of death, compared to having an ITAT score <4 (crude OR 4.80, 95% CI 2.39–9.64). For all vital signs except respiratory rate, a score of 1 (compared to a score of 0) did not predict death. A score of 2 (compared to a score of 0) was a significant independent predictor of death for all vital signs. Beta and data-driven ITAT scores based on multivariable regression are also included but did not suggest that any individual vital sign should have greater weight in determining the ITAT score.
Table 4. Predictors of mortality, using disjoint indicator coding for scores
| ||Individual vital signs||All vital signs|
|Beta||SE||Odds ratio (95% CI)||Beta||SE||Odds ratio (95% CI)||Data-driven score|
|Overall risk score|
|≥4 vs. <4||1.5686||0.3557||4.80 (2.39–9.64)|| || || || |
|1 vs. 0||0.5631||0.4713||1.76 (0.70–4.42)||0.0937||0.4797||1.10 (0.43–2.81)||0.00|
|2 vs. 0||2.8657||1.1096||17.56 (2.00–154.53)||2.1610||1.2186||8.68 (0.80–94.59)||2.00|
|1 vs. 0||0.6078||0.3647||1.84 (0.90–3.75)||−0.0194||0.4405||0.98 (0.41–2.33)||0.00|
|2 vs. 0||2.8338||0.5877||17.01 (5.38–53.83)||2.0289||0.6843||7.61 (1.99–29.08)||2.00|
|1 vs. 0||0.1518||0.4088||1.16 (0.52–2.59)||0.1105||0.4546||1.12 (0.46–2.72)||0.00|
|2 vs. 0||1.3654||0.3957||3.92 (1.80–8.51)||0.5657||0.5238||1.76 (0.63–4.92)||1.00|
|1 vs. 0||1.5612||0.3627||4.76 (2.34–9.70)||1.1582||0.4364||3.18 (1.35–7.49)||1.00|
|2 vs. 0||2.0083||0.5264||7.45 (2.66–20.91)||1.0124||0.6138||2.75 (0.83–9.17)||1.00|
The sensitivity, specificity and LR for data-driven ITAT scores are described in Table 5 and again show little improvement compared to the original ITAT score. The data-driven area under the ROC curve was 0.78.
Table 5. Data-driven score sensitivity, specificity, and likelihood ratios (LR)
- Top of page
In clinical settings, lacking sufficient personnel and training such as KCH, improved inpatient monitoring and triage systems are necessary to identify and treat critically ill paediatric patients prior to deterioration and death. The ITAT scoring system includes a simple objective algorithm that builds on an already existing programme to continue monitoring and triage after a patient has been admitted to the ward. Using this scoring system, healthcare personnel can quickly and efficiently identify children at high risk of death on the ward, allowing clinicians the opportunity to review and intervene on those patients appropriately.
Previously described severity of illness scores have gained widespread acceptance in the developed world, showing improved outcomes following their implementation (Brilli et al. 2007; Hunt et al. 2008; Parshuram et al. 2011). Several limitations, however, inhibit their potential effectiveness in resource-limited settings. One such limitation is that greater performance comes with the cost of greater complexity. The best performing scores such as those developed by Duncan et al. (2006; sensitivity 78%, specificity 95%) and Parshuram et al. (2009; 82% sensitivity, 93% specificity), include many items (up to 16) or are time intensive and clinically subjective (level of consciousness, respiratory effort, airway threat). In addition, several scoring systems include items that are dependent on treatment already provided such as oxygen (Monaghan 2005; Duncan et al. 2006; Egdell et al. 2008; Edwards et al. 2009; Parshuram et al. 2009) intravenous fluids (Duncan et al. 2006; Haines et al. 2006) and nebulised adrenaline (Haines et al. 2006). These items have less utility in settings where their associated treatments are underutilised or not available. Previously reported scoring systems are designed to identify acutely deteriorating patients, often within hours of their arrest or transfer to intensive care. In settings lacking resources for rapid response teams, intensive care, or even routine vital signs, scoring systems may be more appropriately used to guide overall management than to provide triggers for urgent resuscitation.
The greatest strengths of the ITAT score are that it can be carried out quickly and that it requires only a minimal amount of clinical expertise. With only four easily measured items, all of which can be objectively measured using vital sign equipment or direct observation, a single ITAT score assessment can be completed more quickly than previously reported scores. Furthermore, the ITAT score does not include items requiring high levels of clinical knowledge such as evaluations of mental status and respiratory effort, or items that are based on responses to clinical interventions such as IV boluses and oxygen. The ITAT score shows similar characteristics compared to the data-driven ITAT score, suggesting our algorithm would not benefit from more complicated weight adjustment. This overall simplicity allows for the possibility of task shifting this tool to less costly, capable workers who would require less training, allowing personnel with greater experience to focus on more specialised tasks. The ITAT score may be used as a continuous surveillance system with multiple assessments performed each day, allowing clinicians an objective measure of a patient's status over time and opportunities to review and change management plans where appropriate. In settings with high staff and patient turnover, this tool provides some level of continuity between multiple assessments from different clinicians. The threshold score for clinician notification can also be adjusted depending on human resource constraints. Using our data, decreasing the ITAT threshold score to 3 or greater would more than double the number of clinician notifications (unpublished data).
The ITAT score's simplified design, however, also leads to several limitations. One such limitation is that it lacks the accuracy of more complicated scoring systems. Previously validated scores report an area under the ROC curve of 0.86–0.91 (Duncan et al. 2006; Egdell et al. 2008; Edwards et al. 2009; Tucker et al. 2009; Parshuram et al. 2011) compared with the ITAT score ROC of 0.76, suggesting the more complex scoring systems are better able to discriminate patients with poor outcomes. Similarly, sensitivity and specificity are also slightly decreased with our scoring system. Our use of imputation as well as the use of clinician interventions for ITAT scores ≥4 may have decreased our area under the ROC curve, sensitivity and specificity. The use of single imputation may also slightly overestimate precision. Unlike previous scores that aim to recognise deteriorating patients requiring intervention within hours, the ITAT score is designed to predict paediatric death within 2 days. Thus, while children with an elevated score should be urgently evaluated, it should be emphasised that the score is designed as a surveillance tool on the general ward to provide continuous patient monitoring throughout their hospitalisation. As is common at KCH, a substantial amount of missing data resulted from either incomplete or no vital signs being collected by healthcare staff, which may have introduced selection bias into our study. The absence of vital signs was not always at random. Children that died were more likely to have zero vital sign assessments, which was expected as the most critical children were more likely to die immediately following admission to the ward, prior to any vital sign collection. These children may have received vital signs on initial triage in the ‘under five’ clinic prior to admission, although we chose not to include those vital signs in our analysis as they are often incomplete, poorly documented and would not add value in evaluating our inpatient triage tool. Missing vital sign data were also associated with time of day the outcome occurred, with afternoon or night having higher odds of being missing compared with morning (data not shown). The association between ‘time of day’ and missing vital sign data was likely due to fewer on duty staff at night. Influential observations, or single observations that have a significant impact on the regression function, were numerous in our study, which may be indicative of our relatively small sample size. It is possible that missing data and/or influential observations biased our effect estimates and standard errors. A prospective cohort design was not used in this study because while individuals who died most often did so within 48 h of admission, individuals who survived most often had multiple vital sign sets, thus disproportionately affecting score performance. A case–control design avoided this concern.
In conclusion, we have designed and implemented a simplified paediatric severity of illness score (ITAT) capable of recognising children at high risk of death. Once identified as high risk, patients can be triaged for clinician re-evaluation, possible changes in management or increased levels of supervision. Future studies should investigate the best way to operationalise the ITAT programme in similar settings, including significant changes in the ITAT score as trigger for clinician review, patient outcome measures such as mortality and the evaluation of task shifting the ITAT score to health workers with less training.
- Top of page
We would like to thank our assistant program coordinator, Jean Nkhoma and our Vital Sign Assistants, for their valuable contributions to the success of this programme. We would also like to acknowledge Christopher Wiesen and the UNC Odum Institute for help with data management. This work was supported by a grant from Health Empowering Humanity, the National Institutes of Health, as well as the National Institutes of Health Office of the Director, Fogarty International Center, Office of AIDS Research, National Cancer Center, National Eye Institute, National Heart, Blood, and Lung Institute, National Institute of Dental and Craniofacial Research, National Institute on Drug Abuse, National Institute of Mental Health, National Institute of Allergy and Infectious Diseases, and National Institutes of Health Office of Women's Health and Research through the Fogarty International Clinical Research Scholars and Fellows Program at Vanderbilt University (R24 TW007988) and the American Relief and Recovery Act.