Commentary: Advancing measurement of ASD severity and social competence: a reply to Constantino and Frazier (2013)
Article first published online: 12 MAR 2013
© 2013 The Authors Journal of Child Psychology and Psychiatry © 2013 Association for Child and Adolescent Mental Health.
Journal of Child Psychology and Psychiatry
Volume 54, Issue 6, pages 698–700, June 2013
How to Cite
Hus, V., Bishop, S., Gotham, K., Huerta, M. and Lord, C. (2013), Commentary: Advancing measurement of ASD severity and social competence: a reply to Constantino and Frazier (2013). Journal of Child Psychology and Psychiatry, 54: 698–700. doi: 10.1111/jcpp.12065
- Issue published online: 22 MAY 2013
- Article first published online: 12 MAR 2013
- Manuscript Accepted: 29 JAN 2013
The Social Responsiveness Scale (SRS) is currently being used in clinical and genetic studies of autism as both a screener and as a quantitative measure of autistic traits. Our article (Hus, Bishop, Gotham, Huerta & Lord, 2013) assessed the influence of non-specific factors on SRS scores to aid researchers in their interpretations of these scores. In their commentary, Constantino and Frazier (2013) argue that the strong influence of behavior problems on the SRS represents the overlap between neuropsychiatric syndromes, and that behavioral symptoms reflected in SRS scores ‘might actually be caused by the autistic syndrome’ (p. 1). They cite evidence for substantial overlap in genetic associations between ASD, ADHD, and other child psychiatric disorders. Our concern, however, is that if scores on a measure do not distinguish between general behavior problems and autism symptoms, questions about overlap cannot even begin to be answered. Moreover, what was not noted in the commentary was the fact that we found equally large effects of behavior problems and smaller effects of social competence on SRS scores in typical siblings, who did not have ASD.
It is widely recognized that children with ASD have varying levels of general behavior problems, and that many have comorbid conditions, such as ADHD, language delay and intellectual disability (Lundström et al., 2011; Simonoff et al., 2008). However, for researchers seeking to identify causal or risk-related genetic influences, behavioral measures not confounded by these other behaviors are needed to draw conclusions that a particular finding is contributing to specific risk for ASD. Similarly, for researchers who want to evaluate associations between particular regions of the brain and core autism symptoms or social competence, evidence that measures such as the SRS are strongly influenced by general behavior problems would seem to present a problem in identifying the specificity of that region.
Thus, our goal in Hus et al. (2013) was to determine the influences of non-specific factors known to affect scores on other ASD measures (Charman et al., 2007) to provide more informed interpretations of SRS scores. Given the SRS's widespread use in genetic and neurobiological research, we hoped that we could increase the degree to which the SRS measured social competence and ASD symptoms, and thereby extend its utility in drawing associations between behavioral phenotypes and underlying biology. In a previous study, we found this approach was useful in improving the validity of scores on the autism diagnostic observation scale (ADOS). After the introduction of the ADOS, we learned that expressive language level and age (e.g., de Bildt et al., 2004) strongly influenced raw ADOS totals. Through expressive language and age-based algorithms (Gotham, Risi, Pickles & Lord, 2007) and the introduction of severity scores calibrated by the same dimensions, we were able to improve the degree to which the ADOS domain scores represent autism severity within the context of a clinical observation. The calibrated severity scores in the ADOS (Gotham, Pickles & Lord, 2009; Hus, Gotham & Lord, 2012) allow researchers to expand the boundaries of constructs of social-communication deficits and repetitive behaviors and quantify difficulties across an interval scale, rather than simply providing categorical cut-offs – contributing to the need for dimensional measures highlighted by Constantino and Frazier (2013).
We approached the current analysis of the SRS from the same point of view. The SRS is particularly valuable because it provides a range of scores even within a typical population. We wondered if we could better understand what child factors could be controlled to make the SRS a more specific measure of social competence. Recently, Duku et al. (2012) took on a similar task with the SRS, resulting in selection of a subset of 30 items intended to measure social impairments in preschool children, but which still correlated with the CBCL internalizing and externalizing scales, r = 0.65–0.68.
We started by looking for how the SRS related to the social domain of the Vineland Adaptive Behavior Scales, a parent report measure widely used as a measure of social competence (Gillespie-Lynch et al., 2012; Klin et al., 2007). We chose social competence because the SRS is commonly referred to as a quantitative measure of social reciprocity that provides scores that can describe both very competent social skills, as well as the full range of social impairments (see Constantino & Gruber, 2012). As shown in the Hus et al. (2013) article, SRS scores are related to Vineland social domain scores and to current ADI scores, but hardly more than they are to CBCL scores in children with ASD (see Tables 3 & S1). In typical siblings, the SRS was most strongly related to the CBCL, with less strong, but significant relationships to developmental level and social competence. Further analyses, not discussed in the commentary, suggested that the effects of behavior problems were not explained by an influence of behavior problems on social competence in either the children with ASD or their typical siblings (see the online supplement in Hus et al., 2013).
In our search for how to better use SRS scores as a measure of social competence or ASD severity, we also considered the effects of factors that we know influence scores on other autism instruments (e.g., the ADOS) including language level, age, and IQ. Based on clinical experience that there are items on the SRS that parents of less verbally able or more intellectually disabled children often find difficult to answer, we felt it was particularly important to explore how language level and IQ affected SRS scores. After carrying out these analyses and submitting the original article, we were asked by reviewers to analyze other instruments. We did in fact go on to carry out similar analyses for the Autism Diagnostic Interview-Revised (ADI-R; Hus & Lord, 2012) and were able to show that ADI-R scores were also strongly related to age and language level. Because these relationships were predictable and easily quantifiable, ADI-R totals and domain scores could potentially be calibrated, as we have performed for the ADOS, to provide better measures of domain severity (Hus & Lord, 2012).
For the SRS, such a calibration is less straightforward. Behavior problems, age, language level and IQ should all be considered before interpreting scores as a quantitative measure of social competence or ASD. While this is not impossible to do, it means that, as it stands, individuals who receive high-SRS scores may encompass both those with significant ASD symptoms (irrespective of behavior problems), as well as those with high levels of behavior problems (but who may have relatively strong social skills; see Hus et al.(2013), Figure 1). This is not a question of boundaries, but one of precision of measurement.
Measurement factors are recognized as important throughout neurobiology (e.g., Zhu et al., 2009); as mental health researchers, we need to advocate for giving equal weight to measurement issues in describing behavior. Parent factors are widely acknowledged as limitations to using questionnaires over interviews or observational measures. For example, a recent study by Bennett et al. (2012) demonstrated strong effects of maternal depression on the SRS, with a weaker effect on the ADI-R and no effect on the ADOS. We need more studies about the properties of all the ASD measures and we need study from research groups independent of those of us who authored the instruments. Improving measurement is a source of advancement in science; we need to support this endeavor and to support transparency in this approach. Researchers and clinicians are well aware of the enormous variability in ASD. It is logical that measures of ASD symptoms will need to take into account individual differences (e.g., language level, IQ, etc.). To not do this, is to miss an opportunity to understand how best to conceptualize and assess different dimensions of severity in ASD.
S.B., K.G., & C.L. receive royalties for the ADOS-2. C.L. also receives royalties for the ADI-R. Profits from the original study were donated to charity – details of funding for that study and individual researchers are available in the Acknowledgement section of Hus et al. (2013).
Vanessa Hus, 530 Church Street, Ann Arbor, MI 48109-1043, USA; Email: email@example.com
- 2012). Influence of reporting effects on the association between maternal depression and child autism spectrum disorder behaviors. Journal of Child Psychology and Psychiatry, 53, 89–96. , , , , , , & (
- 2004). Interrelationship between Autism Diagnostic Observation Schedule-Generic (ADOS-G), Autism Diagnostic Interview-Revised (ADI-R), and the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) classification in children and adolescents with mental retardation. Journal of Autism and Developmental Disorders, 34, 129–137. , , , , , , & (
- 2007). Efficacy of three screening instruments in the identification of autistic-spectrum disorders. The British Journal of Psychiatry, 191 , 554–559. , , , , , , & (
- 2013). Comments on: The observed association between autistic severity measured by the Social Responsiveness Scale (SRS) and general psychopathology – A response to Hus et al., (2013). Journal of Child Psychology & Psychiatry, 54, 695–697. , & (
- 2012). The Social Responsiveness Scale – Second edition (SRS-2). Torrance, CA: Western Psychological Services. , & (
- 2012). Investigating the measurement properties of the Social Responsiveness Scale in preschool children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 1–9. [Epub ahead of print]. , , , , , , … & (
- 2012). Early childhood predictors of the social competence of adults with autism. Journal of Autism and Developmental Disorders, 42, 161–174. , , , , , , & (
- 2009). Standardizing ADOS scores for a measure of severity in autism spectrum disorders. Journal of Autism and Developmental Disorders, 39, 693–705. , , & (
- 2007). The Autism Diagnostic Observation Schedule: Revised algorithms for improved diagnostic validity. Journal of Autism and Developmental Disorders, 37, 613–627. , , , & (
- 2013). Factors influencing scores on the social responsiveness scale. Journal of Child Psychology and Psychiatry, 54, 216–224. , , , , & (
- 2012). Standardizing ADOS domain scores: Separating severity of social affect and restricted and repetitive behaviors. Journal of Autism and Developmental Disorders, 1–13. [Epub ahead of print]. , , & (
- 2012). Effects of child characteristics on the Autism Diagnostic Interview-Revised: Implications for use of scores as a measure of ASD severity. Journal of Autism and Developmental Disorders, 43, 371–381. , & (
- 2007). Social and communication abilities and disabilities in higher functioning individuals with autism spectrum disorders: The Vineland and the ADOS. Journal of Autism and Developmental Disorders, 37, 748–759. , , , , , & (
- 2011). Autistic-like traits and their association with mental health problems in two nationwide twin cohorts of children and adults. Psychological Medicine, 41, 2423–2433. , , , , , , … & (
- 2008). Psychiatric disorders in children with autism spectrum disorders: Prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 921–929. , , , , , & (
- 2009). Regression models for identifying noise sources in magnetic resonance images. Journal of the American Statistical Association, 104, 623–637. , , , , , , … & (