Presented at the Society for Academic Emergency Medicine annual meeting, Washington, DC, May 2008.
Prospective Evaluation of Two Clinical Scores for Acute Asthma in Children 18 Months to 7 Years of Age
Version of Record online: 2 JUN 2010
© 2010 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 17, Issue 6, pages 598–603, June 2010
How to Cite
Gouin, S., Robidas, I., Gravel, J., Guimont, C., Chalut, D. and Amre, D. (2010), Prospective Evaluation of Two Clinical Scores for Acute Asthma in Children 18 Months to 7 Years of Age. Academic Emergency Medicine, 17: 598–603. doi: 10.1111/j.1553-2712.2010.00775.x
This study was funded by a local internal grant of the Research Centre of the CHU Sainte-Justine. There are no financial arrangements that may represent conflict of interest.
- Issue online: 2 JUN 2010
- Version of Record online: 2 JUN 2010
- Received September 2, 2009; revision received November 29, 2009; accepted December 12, 2009.
- clinical scores;
- emergency medicine
Objectives: The objective was to evaluate the discriminatory ability of two clinical asthma scores, the Preschool Respiratory Assessment Measure (PRAM) and the Pediatric Asthma Severity Score (PASS), during an asthma exacerbation.
Methods: This was a prospective cohort study in an academic pediatric emergency department (ED; 60,000 visits/year) conducted from March 2006 to October 2007. All patients 18 months to 7 years of age who presented for an asthma exacerbation were eligible. The primary outcome was a length of stay (LOS) of >6 hours in the ED or admission to the hospital. Clinical findings and components of the PRAM and the PASS were assessed by a respiratory therapist (RT) at the start of the ED visit and after 90 minutes of treatment.
Results: During the study period, 3,845 patients were seen in the ED for an asthma exacerbation. Of these, 291 were approached to participate, and eight refused. Moderate levels of discrimination were found between a LOS of >6 hours and/or admission and PRAM (area under the receiver-operating characteristic curve [AUC] = 0.69, 95% confidence interval [CI] = 0.59 to 0.79) and PASS (AUC = 0.70, 95% CI = 0.60 to 0.80) as calculated at the start of the ED visit. Significant similar correlations were seen between the physician’s judgment of severity and PRAM (r = 0.54, 95% CI = 0.42 to 0.65) and PASS (r = 0.55, 95% CI = 0.43 to 0.65).
Conclusions: The PRAM and PASS clinical asthma scores appear to be measures of asthma severity in children with discriminative properties.
ACADEMIC EMERGENCY MEDICINE 2010; 17:598–603 © 2010 by the Society for Academic Emergency Medicine
Asthma is the most common chronic pediatric illness. Up to 10% of children are affected at some point in time.1 Over the past years, an increase in the number of emergency department (ED) visits for asthma exacerbation has been reported in North America.2 Most of these visits are for preschool children.2
Adequate evaluation of the severity of asthma exacerbations is important for the initial management of patients, as well as for assessing the clinical response. However, severity is more difficult to assess in children.3 The clinical evaluation and physician’s experience are often not enough to determine the degree of respiratory obstruction.3 To supplement the evaluation, objective and reproducible tools, such as spirometry and peak flow, exist.4 Spirometry measures pulmonary markers such as vital capacity and maximal forced expiratory volume in 1 second.4 Well-trained personnel are required to perform these tests, and they are not routinely available in the ED. Peak flow has been used in the ED to measure the degree of respiratory obstruction.4 This technique is difficult to perform in children less than 6 years of age because of their lack of coordination and comprehension.2
With the absence of objective and standardized criteria, multiple asthma scores have been developed to assess the level of severity of asthma exacerbations.5–17 Of these existent tools, we found only two that have been internally validated and shown to have direct applicability to young children: the Preschool Respiratory Assessment Measure (PRAM)5 and the Pediatric Asthma Severity Score (PASS).7
The PRAM was developed with a prospective cohort of 217 children (3–6 years of age) who presented to the pediatric ED of a university-affiliated center.5 Following multiple logistic regression analyses, five variables were kept in the PRAM: suprasternal retractions, scalene muscle contraction, air entry, wheezing, and oxygen saturation (Table 1). Adequate correlations were found between the PRAM and resistance to forced oscillation, as well as with clinical changes throughout the treatments provided in the ED.5
|Scalene muscle contraction||Absent||Present|
|Air entry||Normal||Decreased at the base||Diffusely decreased||Absent/minimal|
|“Wheezing”||Absent||Expiratory only||Inspiratory and expiratory||Audible without a stethoscope/air entry absent/minimal|
The PASS was essentially derived from the Pulmonary Index (PI) and could be interpreted as a simplified version of the PI.7 Nonetheless, even if the PI was previously validated and used, the PASS has demonstrated benefits over the PI. In 2004, Gorelick et al.7 reported on a prospective cohort of 1,221 patients (1 to 18 years of age) who presented to two urban pediatric EDs. Initially, six clinical variables were evaluated, and only three were kept in the score: wheezing, respiratory effort, and prolongation of expiration (Table 2). The score has shown good discriminatory validity and was correlated with peak expiratory flow rate, oxygen saturation, and hospital admissions. The score also showed a good responsiveness with treatments provided in the ED, which correlated with the clinical changes observed.
|Wheezing (= expiratory sounds heard auscultation)||None/mild||Moderate||Severe or absent due to poor air entry|
|Respiratory work (= utilization of accessory muscles or retractions)||None/mild||Moderate||Severe|
|Prolongation of expiration (= ratio of the length of expiration over inspiration)||Normal/mild prolongation||Moderate prolongation||Severe prolongation|
The quality of these scores confirms the need to conduct a prospective evaluation to determine their respective predictive abilities. The objectives of our study were to evaluate the discriminative ability of these two clinical asthma scores during asthma exacerbations in preschool children who come to the ED.
This was a prospective cohort study. The study was reviewed by the CHU Sainte-Justine, Scientific and Review Board, and informed consent was obtained from the participants.
Study Setting and Population
The study was conducted in the ED of a university-affiliated pediatric hospital with an annual census of 60,000 visits per year. The ED was staffed with board-eligible/board-certified pediatric emergency physicians, pediatricians, registered nurses, and respiratory therapists (RTs) 24 hours per day. Study recruitment occurred from March 2006 to October 2007.
The participants were all patients aged between 18 months and 7 years who presented to the ED for an asthma exacerbation while the recruiting RTs were present. An asthma exacerbation was defined as an episode of wheezing or respiratory distress in a patient with a previous diagnosis of asthma, airway hyperactivity, or previous history of two or more episodes of wheezing treated with inhaled bronchodilators.7,18 Children between 12 and 17 months of age were excluded because of the possible confusion in the etiology of the respiratory symptoms (e.g., bronchiolitis vs. reactive airways disease vs. asthma).
Patients were excluded in the presence of 1) chronic respiratory condition other than asthma (e.g., cystic fibrosis, bronchopulmonary dysplasia, recurrent aspirations); 2) chronic cardiac condition; 3) clinical diagnosis of bronchiolitis; 4) concurrent diagnosis of pneumonia, laryngitis, or whooping cough; or 5) transfer of patients who were currently receiving treatment with bronchodilators from another hospital, ED, or clinic.
All patients were first evaluated at triage by a nurse. If the patient was judged to require bronchodilator treatments within 1 hour, the patient was sent to an observation unit where he or she was further assessed by a RT. Otherwise, the patient was not judged to have an asthma exacerbation. Following this evaluation, the RT discussed with the ED physician the initiation of treatment with bronchodilators. A local standardized protocol was then activated in the delivery and frequency of bronchodilators. This protocol was developed by a local consensus of experts (http://urgencehsj.ca). The decision to prescribe oral corticosteroids and the final disposition of the patient were usually made by the ED physician after evaluation of the patient.19 The physicians were blinded to the PRAM and PASS scores of the recruited patients. Furthermore, the physicians in our ED did not use these scores in their evaluation of the children with an asthma exacerbation.
A priori, six RTs were approached to participate in the study on a part-time basis, and recruitment only occurred while they were in the ED (mostly days and evenings; rarely nights). They all received standardized training regarding the use of the PRAM and PASS systems. Each RT had to rate at least three patients with another RT blinded to the previous assessment before recruiting patients. The interobserver agreement between the RTs was measured a priori using a weighted kappa score for each of the asthma scores (κ = 0.78, 95% confidence interval [CI] = 0.68 to 0.88 for PRAM; and κ = 0.82, 95% CI = 0.76 to 0.87 for PASS).
Four outcome groups were defined: 1) patients with extended ED stays (>6 hours from ED arrival) discharged to home, 2) patients with extended ED stays (>6 hours from ED arrival) admitted to the hospital, 3) patients with standard ED stays (<6 hours from ED arrival) admitted to the hospital, and 4) patients with standard ED stays (<6 hours from ED arrival) discharged to home. A length of stay (LOS) of ≥6 hours in the ED was chosen because in our ED, a patient can typically stay up to 6 hours before admission is deemed necessary. However, if the patient’s condition is severe, she or he can be admitted any time before the 6-hour proxy limit. Sometimes a patient will stay in the ED longer than the usual 6 hours and is eventually discharged to home.
The primary outcome is the combination of the above group 1 (extended ED stay but discharged) groups with 2 and 3 (admitted patients). This classification scheme reflects the utilization of hospital resources by differentiating this combination of patients from the patients who are discharged home from the ED within a standard ED stay (group 4). The secondary outcome is the combination of the above groups 2 and 3 (admitted patients). We differentiate these participants from the discharged patients (groups 1 and 4). This more restricted scheme reflects the physicians’ decision making. Another variable of interest was the clinician’s assessment of the level of severity (mild, moderate, severe, or extreme).
The PRAM and PASS scores were measured by the research RTs on two occasions: upon the initial assessment in the ED and 90 minutes after the initial assessment. A 90-minute time interval was chosen because as per local asthma protocol, many patients either received bronchodilators every 20 minutes ×3 or every 30 minutes ×2 in the first hour of treatment, depending on their initial status. It seemed that 30 minutes after the end of that period was an appropriate time to reassess clinical status. Other clinical and demographic variables were obtained either prospectively (medications given in the previous 24 hours) or by a retrospective chart review (age, time of day, respiratory rate, and heart rate at triage).
Data were entered in an Excel format (Microsoft Corp., Redmond, WA) and analyzed using Stata 8.0 software (StataCorp, College Station, TX). For each score, a receiver-operating characteristic (ROC) curve was constructed to predict a combined LOS of >6 hours in the ED and admission outcome. Model-based ROC curves are presented where curves are plotted from a previously derived logistic equation and are presented as long as the results are similar to those generated with nonparametrically derived area under the ROC curve (AUC) estimates.
To examine the association between the physicians’ assessment of the level of severity and the initial scores, the Spearman’s rank correlation was used. To identify effects between the different variables of interest, analysis of variance was used for the normally distributed variables.
To determine the individual performance of each clinical score, the AUC was estimated as the measure of accuracy. Based on previous studies, we expected an AUC of 0.82 for the PASS.7 There are no such previously reported data on the AUC for the PRAM. On an empirical basis, we expected the score to have a similar AUC of about 0.82. The sample size required to compare AUC values of the asthma scores against a predetermined threshold of 0.70 was 50 patients who stayed <6 hours in the ED and 50 patients who stayed ≥6 hours in the ED and/or admitted were required. A minimum of 50 patients in the least common category was also needed.
During the study period 3,845 patients between the ages of 18 months and 7 years were seen in the ED for an asthma exacerbation. Of these patients, 291 were approached and eight refused to participate. There were no missed eligible patients while the RTs were present. A large proportion (>99%) of the nonrecruited patients was related to the nonavailability of a recruiting RT. Baseline characteristics of the study population were similar to those of the nonparticipants, as reported in Table 3. The overall mean age was 3.4 years, and 63% were boys.
|Characteristic||Included (n = 283)||Not Included (n = 3562)|
|Mean age, yr (95% CI)||3.4 (2.2–4.6)||3.6 (2.3–4.9)|
|Sex, % boys||63||62|
|Mean triage RR (95% CI)||28 (18–52)||27 (20–50)|
|Mean triage O2 saturation, % (95% CI)||96 (93–99)||97 (93–99)|
|Severity as per physician, %|
|Bronchodilators in previous 24 hr (%)||87||NA|
|Inhaled steroids in previous 24 hr (%)||61||NA|
|Admission rate, % (95% CI)||13 (10–16)||11 (9–13)|
|Extended* ED stay, % (95% CI)||19 (15–23)||17 (14–20)|
|Steroids PO within 90 minutes of triage, %||82||NA|
Of the 283 recruited patients, 79 had an ED stay of at least 6 hours: 43 patients had an extended ED stay (>6 hours) but were discharged home (group 1), 12 patients had an extended ED stay (>6 hours) but were admitted to the hospital (group 2), and 24 patients had a standard ED stay (<6 hours) but were admitted to the hospital (group 3). A total of 204 patients had a standard ED stay (<6 hours) and were discharged home (group 4).
Figures 1 and 2 show the distribution of the patients for each level of the PRAM and PASS, respectively. Only a limited number of patients were in the more severe categories of each of the scores, as judged by the EPs in their initial evaluation.
Moderate levels of discrimination were found between the combination of groups 1, 2, and 3 (primary outcome; n = 79), and PRAM (AUC = 0.69, 95% CI = 0.59 to 0.79) and PASS (AUC = 0.70, 95% CI = 0.60 to 0.80), calculated at the start of the ED visit (Figure 3). When the scores were recalculated after 90 minutes of treatment, the levels increased for PRAM (AUC = 0.82, 95% CI = 0.73 to 0.90) but remained similar for PASS (AUC = 0.72, 95% CI = 0.62 to 0.82; Figure 4).
Significant discriminatory levels were found between the combination of groups 2 and 3 (n = 36 patients; secondary outcome; restricted outcome to admission) and PRAM (AUC = 0.86, 95% CI = 0.80 to 0.91) and PASS (AUC = 0.86, 95% CI = 0.83 to 0.89), calculated at the start of the ED visit. When the scores were recalculated after 90 minutes of treatment, the levels increased for PRAM (AUC = 0.91, 95% CI = 0.87 to 0.95) and for PASS (AUC = 0.88, 95% CI = 0.85 to 0.91).
Significant similar correlations were found between the physician’s judgment of severity and initial scores of PRAM (r = 0.54, 95% CI = 0.42 to 0.65) and PASS (r = 0.55, 95% CI = 0.43 to 0.65). These correlations were slightly lower but still maintained after 90 minutes of treatments for PRAM (r = 0.48, 95% CI = 0.33 to 0.60) and PASS (r = 0.47, 95% CI = 0.32 to 0.60). The mean time from the first score assignment to the second was 91.6 minutes (range = 84–109 minutes).
As previously proposed, instead of developing a new score, it is reasonable to evaluate the existing scores’ validity as well as their capacity to asses a significant clinical response change.3 The use of a validated score may lead to better standardized care for patients with asthma and optimal use of resources. Both PRAM and PASS appear to be measures of asthma severity in children that show discriminative properties.
The effect of the predictability of the score is more marked after 90 minutes of treatments, which is in keeping with the nature of asthma exacerbations for children who come to the ED. Often after proper initial treatments, the respiratory status of the patients is greatly improved, and it is often on that status that the clinicians will begin their thinking process of a possible admission for the patients. In spite of a more severe presentation associated with a more severe score level, it is common to eventually send patients home if they improve significantly, even if initially they appeared quite in distress.
For both scores, the ability to predict admission was better than the ability to predict prolonged stay. It is possible to speculate that the physician’s decision about prolonged stay might be more subjective. Therefore, the composite outcome of prolonged stay and admission is still useful for resources prediction; however, it may not be as objective compared to the proposed scores. The correlations between physician judgment and the scores are statistically significant, but they are only fair in terms of the strength of the association.
Our results reinforce a recent study by Ducharme et al.20 that reported that the PRAM was a valid clinical score for assessing acute asthma severity from toddlers to teenagers. In a prospective cohort study of 782 patients (2 to 17 years of age), the performance characteristics of the PRAM were evaluated. The PRAM as measured at triage and after initial bronchodilator therapy showed a strong association with admission. It also showed good internal consistency and good inter-rater reliability for all patients. However, this study is from the same center that had initially developed the PRAM. It remains that their results may not be applicable to other centers with a slightly different approach to asthma management and patient populations. Another limitation of the cited study by Ducharme et al.20 is that only 57% of the children had a second PRAM performed; therefore, a significant amount of data is missing and this limits the interpretation and accuracy of validity. Also, the physicians were not blinded to the recorded PRAM. It is thus possible that their decisions about patient management were influenced by the score. Furthermore, the PRAM had been in place in that institution, and the physicians were already familiar with its use. Overall, the authors concluded that the PRAM had a good performance in all age groups.20
The scores were performed by experienced RTs who had received a priori coaching in the evaluation of the items of the scores. However, the items of each of the scores are routinely evaluated in the day-to-day practice of pediatric emergency medicine. One could argue that using a small pool of specifically trained RTs could misestimate the actual performance of the scores under real world conditions. However, the level of interobserver agreement is very similar to what was reported in the original PASS publication for a larger pool of observers without specific training.7
We did not measure a priori the inter-rater reliability indices for physicians’ decisions. We do recognize that each individual physician has a different threshold for prescribing different treatment options, as well as for admission criteria.
Only a small proportion of patients were recruited during the overnight period. However, we do not believe that their respiratory symptoms would be different, nor that the management of their symptoms would be different. Due to staffing issues, we reported a convenience sample representing approximately 7% of the eligible patients.
Also, for each patient, the same RT reported both scores at time 0 and after 90 minutes of treatments. Because the RTs were not blinded to previous score, it is possible that this may have affected the objectivity of their scoring at the 90-minute evaluation. However, the emergency physicians were blinded regarding of the PRAM and PASS scores attributed at each evaluation by the RTs. Furthermore, in our center no attending physicians were familiar with the use of these scores and their clinical implications. The decisions to give bronchodilators and to admit patients were not based on these scores at the time of the study.
When using the primary outcome scheme, it was possible that “noise” was injected from administrative delays to discharge, e.g., cases for which the treating physician forgot to (or was too busy to) discharge a participant to home in a timely manner. However, in our ED, usually a designated physician per shift is attending to all the asthma-related cases, thus improving the time-related performance of care. In future studies, other variables of interest could be collected for the evaluation of the severity of asthma such as the rate of revisits for the discharged patients and LOS for the admitted patients.
The Preschool Respiratory Assessment Measure and Pediatric Asthma Severity Score clinical asthma scores appear to be measures of asthma severity in children with an asthma exacerbation with discriminative properties.
- 1Summary of recommendations from the Canadian Asthma Consensus Guidelines. CMAJ. 2005; 173(6 suppl):S1–56., , , et al.
- 2Centers for Disease Control and Prevention (CDC), National surveillance for asthma-United Sates, 1980–2004. MMWR Surveill Summ. 2007; 56:1–54., , , et al.
- 18American Thoracic Society. Guidelines to the diagnosis and treatment of asthma. Am Rev Respir Dis. 1987; 136:225–44.
- 19Guide for the comprehensive management of asthmatic children and adolescents [French]. Le Clinicien. 2002; 17:91–106., , , et al.