The predictive value of the ages and stages questionnaire in late infancy for low average cognitive ability at age 5

Abstract Aim This retrospective, longitudinal study examined the predictive value of the ages and stages questionnaire (ASQ) in late infancy for identifying children who progressed to have low cognitive ability at 5 years of age. Methods The ASQ was performed on 755 participants from the Irish BASELINE birth cohort at 24 or 27 months of age. Intelligence quotient was measured at age 5 with the Kaufmann Brief Intelligence Test, Second Edition, and low cognitive ability was defined as a score more than 1 standard deviation below the mean. The ASQ’s predictive value was examined, together with other factors associated with low cognitive ability at 5 years. Results When the ASQ was performed at 24 or 27 months, the overall sensitivity for identifying low cognitive ability at 5 years was 20.8% and the specificity was 91.1%. Using a total score cut‐off point increased the sensitivity to 46.6% and 71.4% at 24 and 27 months, but specificity fell to 74.1% and 67.2%, respectively. After adjusting for ASQ performance, maternal education and family income were strongly associated with cognitive outcomes at 5 years. Conclusion The ASQ did not detect the majority of children with low cognitive ability at age 5. Alternative methods need investigation.

Defining this level of cognitive ability can be difficult. The International Classification of Diseases, 11th Edition clearly defines a disorder of intellectual development as one that is characterised by significantly below average intellectual functioning and adaptive behaviour that are approximately two or more standard deviations (SD) below the mean on appropriately normed, individually administered standardised tests. 8 The intellectual functioning of some children is substantially below the population average, but they do not meet these criteria. They remain in a grey area between average intellectual functioning and disordered intellectual development. Children with more significant intellectual developmental disorders display marked deficits that are likely to be detected during routine developmental screening. However, children in the grey area may not display overt signs during infancy. Their cognitive difficulties may go unrecognised until they experience repeated failures or behavioural problems at school. 1,7,9 Early interventions are best focused on the first five years of life, a fundamental period in cognitive development, and the aim of routine developmental screening is to identify infants who would benefit from such interventions. 10 The American Academy of Pediatrics recommends that parent-reported measures should be considered during routine developmental screening and the Ages and Stages Questionnaire (ASQ) is one example. 11 The ASQ is a parent-reported questionnaire designed to discriminate between children with developmental delays and typical development. 12 The National Steering Group for the Revised Child Health Programme has recommended its use during the Irish 24-month public health nursing assessment. 13 Studies have suggested that the ability of the ASQ to predict below average childhood intelligence quotient (IQ) depends on the cohort and the cut-offs used. 14,15 The ASQ has high specificity at a general population level and may effectively identify children not at risk of neurodevelopmental disorders. 16 A significant shortcoming of many developmental screening tests is that they do not adequately account for the social environment to which the child will be exposed. 17 Charkulak et al found that the performance of the ASQ for identifying children with low IQ is dependent on the maternal education level. 18 This study examined the predictive value of the ASQ performed in late infancy for identifying those who progress to have low average cognitive ability at 5 years of age. We also assessed whether the 24-month or 27-month ASQs would have a better predictive value if an optimal cut-off, based on the total score, was used. Finally, we examined other important factors associated with lower than average cognitive ability in early childhood.

| Participants
The study population consisted of mothers and infants from the Irish Cork BASELINE Birth Cohort Study, which started in 2008. 19 BASELINE recruited 1583 of the 2183 participants (73%) from the Screening for Pregnancy Endpoints study, a multi-centre prospective study of low-risk, primiparous women. The other 600 were recruited from the postnatal wards of Cork University Maternity Hospital. The subjects were followed up at 2, 6, 12 and 24 months and 5 years. Intelligence Test, Second Edition (KBIT-2) at the age of 5. These 755 participants formed the study cohort ( Figure S1).

| Measures
The ASQ is a parent-completed questionnaire with 30 items covering 5 development domains: communication, gross motor, fine motor, problem solving and personal social skills. 12 The score is based on the parental response, which may be "yes" (10 points), "not yet" (0 points) or "sometimes" (5 points), to a statement about the child's activity. Parents are asked to try the activities with their child, if necessary, before answering. Total scores are calculated for each domain and missing answers may be replaced by the mean of the other answers, if there are no more than 2 missing items per domain. Age-specific statistically derived cut-off points have been established for onward referral. The ASQ has not been validated in Ireland, and onward referral for further professional assessment is made based on an age-specific score, in any domain, that falls at least 2 SD below the mean in the reference population. 12 The BASELINE study mothers received an age-appropriate ASQ-3 and were asked to complete it before their next assessment at the research facility: 468 received the 24-month ASQ and 287 received the 27-month ASQ. The cut-offs for onward referral are outlined in Table S1.
The KBIT-2 was administered by trained research nurses to measure cognitive ability at 5 years of age. It comprises three subtests: verbal knowledge and riddles, which measure verbal intelligence, and matrices, which measures non-verbal intelligence. The verbal and non-verbal scores are combined to provide a composite IQ. The Kaufmann test has been shown to be a reliable and valid measure of IQ in children and adults aged 4-90 years. 20 Children who scored more than 1 SD below the cohort mean were categorised as "Low Cognitive Ability" and those who scored above this as "Average or Above Cognitive Ability".

Key Notes
• This study assessed whether using the Ages and Stages Questionnaire at 24 or 27 months of age identified children with low average cognitive ability at 5 years.
• The overall sensitivity and specificity of the ASQ for identifying children with low average cognitive ability at 5 years were 20.8% and 91.1%, respectively.
• Current developmental screening will not detect the vast majority of children who commence education with low average cognitive ability.

| Child, maternal and sociodemographic variables
Child characteristics included gender and weeks of gestation.

| Statistical analysis
The statistical analysis was conducted using R version 4.0.3 (R Foundation, Vienna, Austria). The study population was described using counts and percentages for categorical data and means and standard deviations for continuous data. T-tests were used to compare the mean ASQ scores between the cognitive ability groups.
The predictive value of the ASQ was examined by calculating its sensitivity, specificity, positive and negative predictive values and their 95% confidence intervals (CI). Previous studies have reported sensitivities ranging from 28% to 77% for predicting low cognitive performance, with results varying by the cohort tested and cut-offs used. 14,15 The pROC package in R was used to generate receiver operating characteristic curves and calculate optimal cut-off points for the total ASQ score. 21 Univariable and multivariable logistic regression was used to examine child, maternal and socioeconomic predictors of low cognitive ability at 5 years. The findings are reported in line with the Strengthening the Reporting of Observational Studies in Epidemiology guidelines. 22

| Description of sample
The study sample consisted of 755 children who underwent the ASQ at either 24 or 27 months and the KBIT-2 at 5 years. Their characteristics are compared to those who were not included due to missing data or attrition ( Table 1). Children in the study sample were more likely to have a higher birthweight, have older mothers with more schooling, coming from higher income backgrounds and have higher ASQ scores. The ASQs were completed at a mean age of 25.1 ± 1.1 months, but the results are presented separately for 24 and 27 months, as the cut-off points and score distributions differ between the 2 questionnaires. The Kaufmann test was completed at a mean age of 60.8 ± 1.7 months. The mean composite IQ score was 104.3 and 13.4% scored more than 1 SD below this point.
The results of the univariable and multivariable logistic regression analyses are shown in Table 3. After the data were adjusted for maternal and socioeconomic factors, those who failed at least one ASQ domain were 2.9 times more likely (adjusted odds ratio (AOR)

| DISCUSS ION
Our main aim was to examine the predictive value of the ASQ in late infancy for identifying children in the general population who progress to have low cognitive ability at 5 years of age. When the onward referral criterion of failing in a single domain was used, the overall sensitivity was 20.8%. The specificity, as previously documented, 16 was 91.1%, making it a useful tool for identifying those not at risk. While the ASQ may identify children with significant domain-specific developmental delay, for the purpose of identifying children with low cognitive ability in childhood, the ASQ at 24 or 27 months has an unacceptably low sensitivity, missing 80% of children who may benefit from earlier intervention.
The second aim was to determine whether an optimal cut-off point, based on the total ASQ score, could improve the tool's predictive value. Total ASQ scores are not currently used for onward referral, but studies suggest this may be a useful additional measure. 15,23 Using optimal cut-off points of 249.5 and 244.5 on the 24-month and 27-month ASQs improved sensitivity to 46.6% and 71.4%, respectively, but reduced specificity. In population-based screening, a careful balance must be struck between the sensitivity and specificity of a test. This improved sensitivity would mean that almost half of all children who progress to low cognitive ability at age 5 could be identified at 24 months. However, the accompanying reduction in specificity would mean that at 24 months approximately 1 in every 4 TA B L E 1 Characteristics of children in study population compared to those in BASELINE cohort not included in study  This has significant ethical and societal implications. Labelling a child incorrectly as at risk of low cognitive ability may result in stigmatisation or may lead to lower expectations of ability which can adversely affect evolving capabilities. 24 Additionally, a criterion for implementing any population-based screening programme is that there must be an available course of action with regard to further diagnosis, treatment or intervention. 25 Implementing this cut-off could threaten to overwhelm the finite resources dedicated to early intervention programmes for at risk children. Implications would be similar with implementation of the 27-month total score cut-off.
The third aim was to examine the child, maternal and socioeconomic characteristics associated with low cognitive ability at 5 years.
The findings were consistent with studies that have demonstrated strong, consistent associations between families' socioeconomic characteristics and children's cognitive outcomes. 18 The odds of low cognitive ability were significantly higher for children whose mothers had lower levels of education and who were from lower income backgrounds. This finding again highlights the pivotal role socioenvironmental factors have in the cognitive development of a child.
This study investigated one potential screening method, the ASQ, for the purpose of early identification of children who progress to have low cognitive ability. It was found that, as it is currently used, this tool will not identify the majority of children with low cognitive ability at school entry. Other tools exist for identifying children at Our study has limitations. The research question focussed on the use of the ASQ at a single time point in late infancy, but the authors acknowledge that developmental assessment should be a continuous process performed at multiple time points throughout infancy and childhood. The outcome was cognitive ability at age 5, but it is well recognised that the social and emotional competence of a child at this age are equally important determinants of outcomes in childhood. 29 Cohort studies are subject to selection bias, and those who participate tend to systematically differ from those who do not.