Individual risk assessment tool for school‐age asthma prediction in UK birth cohort

Summary Background Current published asthma predictive tools have moderate positive likelihood ratios (+LR) but high negative likelihood ratios (−LR) based on their recommended cut‐offs, which limit their clinical usefulness. Objective To develop a simple clinically applicable asthma prediction tool within a population‐based birth cohort. Method Children from the Manchester Asthma and Allergy Study (MAAS) attended follow‐up at ages 3, 8 and 11 years. Data on preschool wheeze were extracted from primary‐care records. Parents completed validated respiratory questionnaires. Children were skin prick tested (SPT). Asthma at 8/11 years (school‐age) was defined as parentally reported (a) physician‐diagnosed asthma and wheeze in the previous 12 months or (b) ≥3 wheeze attacks in the previous 12 months. An asthma prediction tool (MAAS APT) was developed using logistic regression of characteristics at age 3 years to predict school‐age asthma. Results Of 336 children with physician‐confirmed wheeze by age 3 years, 117(35%) had school‐age asthma. Logistic regression selected 5 significant risk factors which formed the basis of the MAAS APT: wheeze after exercise; wheeze causing breathlessness; cough on exertion; current eczema and SPT sensitisation(maximum score 5). A total of 281(84%) children had complete data at age 3 years and were used to test the MAAS APT. Children scoring ≥3 were at high risk of having asthma at school‐age (PPV > 75%; +LR 6.3, −LR 0.6), whereas children who had a score of 0 had very low risk(PPV 9.3%; LR 0.2). Conclusion MAAS APT is a simple asthma prediction tool which could easily be applied in clinical and research settings.

Parentally reported, either (a) physician-diagnosed asthma and wheeze in the previous 12 months or (b) more than 3 wheeze attacks in the previous 12 months.

| Current eczema (age 3 years)
Parentally reported, answered positively to both "Did the doctor ever tell you that your child had eczema?" and "Does your child still have eczema?"

| Allergic sensitisation (age 3 years)
Skin prick tested mean weal diameter at least 3 mm greater than the negative control to any of the allergens tested (house dust mite, cat, dog, grasses, moulds, milk and egg [Bayer, Elkahrt, IN, USA]).

| Statistical analysis
Relevant variables were assessed in univariate logistic regression models and these variables were entered into a backward stepwise multivariable logistic regression model. Predictors that improved the model fit (Akaike and Bayesian Information Criterion, AIC and BIC) were included in MAAS APT. A forward stepwise multivariable logistic regression model was used to assess the robustness of the backward stepwise selected model. Any differences in the variables selected were resolved using the discriminative ability of the model, model fit (AIC, BIC) and clinical interpretation of the results.
We derived a simple scoring system by rounding up regression coefficient to the nearest integer. The total score was calculated for each subject. We assessed the discriminative ability of this model using a receiver operating characteristic (ROC) curve and the area under the curve (AUC

| Study population
Out of 1184 participating families recruited during pregnancy, 995 children completed follow-up at age 3 years, and 916 (92%) had primary health care records data extracted. Of these, 336 children had documented and confirmed wheeze in their health care record by age 3 years. These children were included in the model development ( Figure S1). Asthma at school-age (8 or 11 years) was present in 117/336 (35%) children.

| Development of the model
Of the 22 predictors measured at age 3 years which were assessed in the univariate logistic regression analysis, 11 were significantly associated with asthma at school-age (Table S2. P ≤ 0.01). When the predictors were entered in a multivariable logistic regression model  (Table 1). When a forward stepwise selection procedure was performed on the 22 predictors, the same five predictors were selected. Combining these predictors into an asthma predictive tool (MAAS APT, Table 1) allows children to score a minimum of zero, up to a maximum of five points.
A total of 281 (84%) children had complete data available on all 5 predictors at age 3 years and were used to test the MAAS APT.
For the children who scored 5 on the MAAS APT, all three had asthma (100% risk, Table 2). Almost half of children with school-age asthma had a score of ≥3 at the age of 3, with a false positive rate of only 7.4% (Table S3). Children with a score ≥3 had a greater than 75% risk of school-age asthma (+LR 6.3, −LR 0.6; OR: 11.0, [95%CI 5.6-21.7], P < 0.001). Therefore, we defined a MAAS APT score ≥3 as "High risk" (Table 1).
Almost half of the children had a score of 1 or 2, and the risk of asthma at school-age was difficult to predict in these children (+LR 0.9, −LR 1.1) Of the 138 children that fell within this group, 29.7% had asthma at school-age, accounting for 45% of the asthma cases (Tables 2 and S3). Therefore, we defined a MAAS APT score of 1-2 as indeterminate risk.

| Multiple imputation and internal validation
Fifty data sets, each with 336 patients were created. The regression coefficients (Table S4) from the final model selected in the original complete cases analysis, calculated from the pooled analysis of the 50 imputed data sets were similar to those from the model in the original data set. The calibration values (Table S5) indicate the degree of agreement between observed outcomes and predictions.

| Summary of results
We developed a simple prediction tool for asthma at school-age amongst children who had physician-confirmed wheeze in the first 3 years of life, within the context of a population-based birth cohort.
In developing the MAAS predictive tool, we assessed severity, frequency and triggers of wheeze and cough, other atopic history, family history, smoking exposure, atopic status and demographic details.
This final model comprises only 5 predictors, resulting in a score between 0 and 5. The tool provides moderately high +LR (6.3) and a reasonably low −LR (0.6) in children with a high score (≥3,~20% of all wheezy children at the age of three). Conversely, for children who had confirmed wheeze, but without the presence of any predictive features stated in MAAS APT (score = 0), the risk of asthma at school-age was very low (<10%; LR 0.2).Therefore, for children with GP-confirmed wheeze by age 3 years, the MAAS APT score performs well for those with high scores and those with low scores.
Amongst children with confirmed wheeze at age 3 years, half had a MAAS APT score of 1 or 2; and because of this poor discriminative ability, we termed this an "indeterminate" risk group. This large "indeterminate" group accounted for almost half (45%) of the children who developed school-age asthma. We speculate that within this "indeterminate" group, ongoing and future environmental factors may determine the persistence of wheeze in these children who had wheezed in early life. Furthermore, it is possible that in this "indeterminate" group of children, development of school-age asthma is potentially preventable with an environmental modification, and future studies of secondary prevention may wish to target this group.

| Comparison with previous prediction models
Since the development of API, 9 studies have been carried out to validate its reliability in South America 21 and UK populations. 20 These results were comparable to the original paper. However, it is recognized that due to its moderate positive likelihood ratio but high negative likelihood ratio, the applicability in the clinical setting is limited. 14,20,21 Subsequently, a number of other studies have tried to develop a tool that is more discriminative than the API.
The PIAMA risk score developed from a Dutch birth cohort was based on 8 predictors. 12 The AUC (0.74) was similar to that of MAAS APT, but PPV in those with a high score was only 50% and the sensitivity was extremely low (7%). 12 External validation of PIAMA risk score showed good agreement, and a modified score was developed. 15 However, the high-risk group had lower sensitivity (11.6%) and PPV (26.4%) compared to that of MAAS APT (47% and 75%, respectively). 15 A Norwegian group developed a predictive scoring system to predict asthma at the age of 10 years based on the severity of obstructive airway disease at the age of 2. 13 However, the suggested cut-off value by the authors gave a PPV of only 54.3% with moderate +LR (4.3) and −LR (0.6). A UK-based study also developed a score to predict persistence of wheeze at age 10 years in those that wheezed in the first 4 years of life. 11 This score included 4 factors collected at different time-points in early life (age 1, 3 and 4 years). A maximum score (4), using this tool, had a T A B L E 1 Backward stepwise multivariable logistic regression of predictors recorded at age 3 years, for possibility of developing asthma at school-age | 295 moderately high +LR (7.9) for persistence of wheeze, but the negative likelihood ratio was also high (0.9). In addition, the need of obtaining clinical information at age 1, 3 and 4 years also markedly reduces its ease of use and clinical applicability.
Another UK-based study of 1226 children with clinically significant wheeze or cough at age 1-3 years who presented to the primary-care physician were analysed in the Leicestershire Respiratory Cohort 5 years later, 10 and an asthma prediction tool was developed. The tool consists of 10 questionnaire based predictors. In addition, the development of the tool was solely based on parentally reported symptoms, and no extraction of GP records was carried out. The discriminative ability in terms of AUC was comparable to the MAAS APT. The high-risk group gave a higher +LR than MAAS APT (9.4 vs 6.3, equivalent to~5% difference in clinical probability 14 ), but a much lower sensitivity (22% vs 47%) and a higher −LR (0.8 vs 0.6). The tool simply involves administration of a questionnaire with 10 questions. However, some questions use rather subjective terms including "a little" and "a lot." In comparison, MAAS APT is designed to be succinct for clinical applicability and includes 5 yes/no questions. This avoids excessive subjective measurement and potential inter-observer discrepancies or reporting bias. The addition of SPT to determine sensitisation contributes to the predictive ability of the tool. In our cohort, the addition of SPT in MAAS APT identified a further 19 children who were at high risk of school-age asthma, compared to MAAS APT without SPT as a predictor. Of these 19 children, 14 had asthma at school-age. This accounts for 33% of all children who had school-age asthma who were in high-risk group.
It is also of interest that in this cohort parental history of asthma/atopy (ie, heritable components) was not statistically significant in the prediction of school-age asthma.
Although MAAS APT showed slightly better predictive abilities reflected in AUC, +LR and −LR when compared to previously developed asthma predictive tools, the increase in the risk of school-age asthma is only moderate for clinical interpretation, and the risk of misclassification remains significant. A proposed simplification for interpretation of likelihood ratios into clinical probability was previously published. 16 According to this, a +LR of 6.3 in high-risk group (score ≥3) corresponds to increased probability of school-age asthma of approximately 35% and a +LR of 0.2 in the low-risk group (score = 0) corresponds to a decreased probability of asthma of about 30%.

| Strengths and limitations
Unlike many of the established asthma prediction tools, MAAS APT was developed within a cohort of children who had GP-confirmed diagnosis of wheeze. In the current cohort, 28% (94/336) of children who had GP-confirmed wheeze, on questioning the parents denied that the child had ever wheezed by the age of 3 years. The poor correlation between parentally reported wheeze and physician-confirmed wheeze is well recognized. 26,27 Given the discrepancies between parentally reported and physician-confirmed wheeze, a    To make the current method more robust and minimize "overfitting," optimal estimates and 95% CIs from the backward stepwise logistic regression model were obtained by using the bootstrapping technique. This technique was also used to internally validate the prediction model, including the model selection process. This method of internal validation may be better suited over other methods of cross-validation with the given sample size. 28,29 This method "fine- We recognize that SPTs may not be readily available in primary care. However, skin tests are available in most secondary care settings, are less invasive than serum eosinophil counts, and our data show that they add value to the MAAS APT.

| CONCLUSION
MAAS APT may be a useful tool which combines clinical history and objective measures in predicting future risk of asthma in both clinical and research settings. It provides a simple assessment for asthma risk at school-age in young children. Although internally validated, further external validation of this tool is needed.