Psychometric Properties of a New Measure to Assess Autism Spectrum Disorder in DSM-5

Authors


An earlier version of this article served as Ms. Monaghan's MA thesis.

Correspondence concerning this article should be addressed to Frederick L. Coolidge, Department of Psychology, 1420 Austin Bluffs Pkwy., University of Colorado, Colorado Springs, CO 80918. Electronic mail may be sent to fcoolidg@uccs.edu.

Abstract

This article presents preliminary psychometric properties of a new 45-item scale, the Coolidge Autistic Symptoms Survey (CASS), designed to differentiate between children within the autism spectrum (including Asperger's Disorder) and purportedly normal children, in anticipation of DSM-5 changes, in which a single diagnostic category is proposed: autism spectrum disorder. The final sample (N = 72) consisted of 19 children diagnosed with Asperger's Disorder, 19 children who were considered loners by their parents (without an autism diagnosis), and 34 purportedly normal children. The CASS and the 200-item, DSM-IV-TR aligned, Coolidge Personality and Neuropsychological Inventory were completed by a parent. The CASS had excellent internal scale reliability (α= .97) and test–retest (r = .91) reliability. ANOVA revealed the CASS was able to discriminate significantly among the 3 groups of children. Further research with the CASS appears warranted.

The purpose of this article is to present the preliminary psychometric properties of a new measure designed to assess the proposed revisions to the diagnosis of autism in the forthcoming fifth version of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2011). The proposed new single diagnosis, autism spectrum disorder (ASD), will apparently subsume the previous DSM-IV-TR (American Psychiatric Association, 2000) diagnoses for autistic disorder, Asperger's disorder, childhood disintegrative disorder, and pervasive developmental disorder not otherwise specified (NOS). The American Psychiatric Association (2011) has stated that the rationale for the change is that, because autism is best defined by a common set of behaviors, it may be best represented by a single diagnostic category. Further, an individual's clinical presentation is likely to vary by a number of clinical specifiers, such as the severity of the disorder, social communication and social interaction difficulties, and repetitive patterns of behavior.

The current DSM-IV-TR diagnosis for autistic disorder assesses three major domains: qualitative impairments in social interactions; qualitative impairments in communication; and restricted, repetitive, and stereotyped patterns of behavior, interests, and activities. The current diagnosis for autistic disorder also requires delays or abnormal functioning in at least one of the following areas prior to 3 years old: social interaction, language as used in social communication, or symbolic or imaginative play. The proposed DSM-5 change for ASD is expected to combine the first two domains into a single category of social and communication deficits. The rationale is that the two are “inseparable” and should be considered as a single set of symptoms that vary with social context and other environmental variables (American Psychiatric Association, 2011). The DSM-IV-TR criteria for Asperger's disorder also required the absence of language delays. In the proposed new category of ASD, it is assumed that delays in language are neither unique to the autism spectrum nor universal to it; thus, it will be assumed in DSM-5 that delays in language or the absence of delays in language will affect an individual's clinical presentation but should not be a defining diagnostic autistic spectrum criterion. The DSM-5 will also make the assumption that ASD is a neurodevelopmental disorder that expresses itself in infancy or early childhood, but it may not manifest itself fully initially because there may be minimal social demands made of children early and because of varying levels of help and support from parents and caregivers.

Another diagnostic issue for the current autism spectrum is differentiating between syndromes primarily in the higher functioning end of the autistic spectrum (e.g., Asperger's disorder and high-functioning autism, although the latter is not mentioned in DSM-IV-TR) from schizoid personality disorder. DSM-IV-TR states that the differential diagnosis for the autism spectrum from the schizoid personality disorder is problematic, as the relationship between those two disorders is “unclear” (p. 83), and there is a “great difficulty” (p. 696) in their differential diagnosis. Furthermore, DSM-IV-TR notes that schizoid personality disorder can first manifest during childhood and contains overlapping symptomatology of diagnoses in the autism spectrum. Some of the DSM-IV-TR overlapping criteria are the lack of close peer-group relationships (Asperger's disorder, schizoid personality disorder) and the fact that the child does not seek others for enjoyment in activities and appears to prefer solitude (Asperger's disorder, schizoid personality disorder). Wolff (1995) identified comorbidities of children with schizoid personality disorder as being “very common” (p. 36) and misdiagnoses commonplace. The essential problem in the differential diagnosis is that the qualitative impairments in social interactions and the restricted patterns of behavior, interests, and activities of individuals with a diagnosis of autism, whose cognitive and intellectual functioning is relatively unimpaired, share a pronounced overlap in the aforementioned symptoms with those who display schizoid personality disorder traits. However, marked differences in the diagnoses arise in the context of stereotyped behavioral patterns (e.g., intense focus of a single subject and repetitive behaviors), which are seen in children within the autism spectrum, but not in schizoid personality disorder. Yet, there exist clinical suspicions that people with schizoid personality disorder prefer or choose to be alone, whereas people with the less severe forms of the autism spectrum are suspected to have an underlying neurological impairment that results in their aloneness (Wolff, 1995). Thus, a proper diagnosis, especially during crucial developmental milestones, is essential for the proper care and treatment of these individuals.

Wolff (1995) also asserted that the schizoid personality diagnosis has been used to characterize children with those specific patterns of behavior (i.e., loners) as early as 1926, and she noted that prevalence rates are as high as 12% in community samples of children. However, in a recent nationwide study of prevalence rates of personality disorders, Grant et al. (2004) reported a more conservative prevalence rate of about 3% in the population. It seems plausible that because both ASD and schizoid personality disorder can manifest in childhood (Coolidge, Thede, & Jang, 2001) and given the considerable overlap in diagnosable criteria, it may sometimes be difficult to clinically discriminate between the two disorders, thereby making the treatment plans difficult for either disorder in some children. Thus, a measure that would differentiate the proposed ASD from schizoid personality disorder traits would be of diagnostic value.

The DSM-IV-TR also notes that inattention and hyperactivity are frequently seen in Asperger's disorder and that attention deficit hyperactivity disorder (ADHD) is often diagnosed prior to a diagnosis of Asperger's disorder. Lee and Ousley (2006) found that 83% of children classified with either autistic disorder (= 39), Asperger's disorder (= 5), or pervasive developmental disorder not otherwise specified (= 8) also fit the DSM-IV-TR criteria for ADHD. Furthermore, they suggested that even though ADHD symptoms are present in children with Asperger's disorder and autism, the underlying behaviors may actually be the result of the pervasive developmental disorder. The proposed DSM-5 changes to the autism spectrum do not yet include the issues surrounding the differential diagnosis of ADHD and the autism spectrum; however, it does list, under the restricted, repetitive patterns of behavior domain, the possibility of hyperreactivity or hyporeactivity to sensory aspects of the child's environment.

The DSM-IV-TR also notes that Asperger's disorder can be associated with depressive disorders. Stewart, Barnard, Pearson, Hasan, and O'Brien (2006) noted that depression is commonly found in individuals with autism and Asperger's disorder. They discussed a need for a depression measurement tool, specifically for the screening of depression in individuals with autism and Asperger's disorder, because the existing measures (e.g., the Hamilton Depression Rating Scale, the Beck Depression Inventory, etc.) do not account for the lack of expression of emotions seen in individuals with autism and Asperger's disorder.

As noted earlier, high-functioning autism (HFA) is not mentioned in DSM-IV-TR, and the issue of whether HFA is different from Asperger's disorder is a point of debate (e.g., Thede & Coolidge, 2007), especially in light of the proposed changes to the DSM. Currently, the literature regarding HFA shows much similarity with the symptoms of Asperger's disorder, including higher functioning verbal skills. Despite their spared cognitive skills, children with Asperger's disorder may exhibit symptoms of executive function deficits of the frontal lobes (e.g., Ozonoff, Pennington, & Rogers, 1991). Thede and Coolidge (2007) found that a group of children diagnosed with Asperger's disorder (= 16) was not significantly different on a parent-rated measure of executive function deficits from a group of children (= 15) diagnosed with HFA; yet, both groups scored significantly worse on the measure than a control group (= 31) with a large effect size.

In a previous study, Thede and Coolidge (2007) sampled the extant literature on the entire autism spectrum symptomatology and created a 44-item survey that was specifically designed to differentiate the symptoms of less-severe forms of autism, such as Asperger's disorder and high-functioning autism from nonclinical children. This survey was shown to have high internal reliability (Cronbach's α = .98; = 88) and excellent test–retest reliability (= .93); however, it could not differentiate between children with Asperger's disorder and children with a diagnosis of high-functioning autism. It did significantly differentiate between those two groups of children and a control group of children (without those diagnoses and no presenting clinical diagnosis). In the anticipation of the proposed DSM-5 changes, an additional item was added to the 44-item survey, and it was renamed the Coolidge Autistic Symptoms Survey (CASS).

In this study, it was hypothesized that (a) the new 45-item CASS would exhibit good internal reliability and test–retest reliability, (b) the CASS would differentiate significantly among children classified as loners (schizoid personality disorder symptoms) by their parents, children diagnosed with Asperger's disorder, and a group of controls without a diagnosis of either, (c) the Asperger's group would score significantly higher than the other two groups on a measure of ADHD, (d) the Asperger's group would score significantly higher than the other two groups on a measure of depression, and (e) the Asperger's group would score significantly higher than the other two groups on a measure of executive function deficits. Finally, a principal components analysis was planned to explore the underlying component structure of the CASS.

Method

Participants

Participants in this study were 44 boys and 28 girls ranging in age from 3 to 16 years old (= 8.8 years, SD = 4.2 years). The parents of these participants were recruited from friends and families of students enrolled in psychology courses at a Western university. An initial demographic survey assessed whether the children in the study had been diagnosed with Asperger's disorder by a mental health professional (PhD, MD, or PsyD) and a group of children whose parents considered their children loners but had never been given any diagnosis by a mental health professional. The final sample (= 72) consisted of 19 children diagnosed with Asperger's disorder by a mental health professional, 19 children who were considered loners by their parents but had never been diagnosed as having autism, and 34 children in a control group who had never warranted any psychological diagnosis according to parental reports. This study was conducted with the approval of the University's Institutional Review Board.

Materials and Procedure

The measures used in this study included the 45-item CASS and a parent-as-respondent, 200-item scale, the Coolidge Personality and Neuropsychological Inventory for Children (CPNI; Coolidge, Thede, Stewart, & Segal, 2002). The CASS was developed specifically as a clinical aid for diagnosing symptoms within the proposed DSM-5 single diagnostic category, ASD. The CPNI contains scales measuring 12 childhood personality disorder traits, anxiety and depression scales, a scale measuring the official criteria in the DSM-IV-TR for ADHD, and a scale measuring executive function deficits. The CPNI has extensive evidence for its reliability and validity (Coolidge, DenBoer, & Segal, 2004; Coolidge, Segal, Coolidge, Spinath, & Gottschling, 2010; Coolidge, Segal, Stewart, & Ellett, 2000; Coolidge, Starkey, & Cahill, 2007; Coolidge, Thede, & Jang, 2001, 2004; Coolidge, Thede, & Young, 2000, 2002; Thede & Coolidge, 2007), and norms for the CPNI are based on 780 purportedly normal children from ages 5 to 17 years old (30 boys and 30 girls from each age group from 5 to 17 years old; Coolidge, Thede, Stewart, et al., 2002). The CPNI manual (Coolidge, 2000) notes that the inventory has been validated empirically to be appropriate for children as young as 3 years or as old as 19 years. Specifically, portions of the CPNI that were used for this study were the 7-item Schizoid Personality Disorder scale (Cronbach's α = .40, test–retest = .61), the 18-item ADHD scale (α = .91, = .83), the 7-item Depressive Personality Disorder scale (α = .71, = .78), and the 8-item Executive Function Deficits scale (α = .86, test–retest = .81; see Coolidge, Thede, Stewart, et al.,2002)

Packets containing the CASS, CPNI, a demographic information sheet, and an informed consent form were hand-delivered or mailed to parents. The CASS items were answered on a 1 (Hardly Ever or Never) to 4 (Frequently) scale, and the sum across the 45 items served as the dependent variable in subsequent CASS analyses. The CASS items provide coverage for the current DSM-IV-TR three domains for autistic disorder: qualitative impairments in social interactions; qualitative impairments in communication; and restricted, repetitive, and stereotyped patterns of behavior, interests, and activities. The CASS also provides coverage for the proposed DSM-5 new domain, persistent deficits in social communication and social interaction, and its three specific diagnostic criteria (i.e., deficits in social–emotional reciprocity, nonverbal communication, and developing and maintaining relationships). The CASS also provides coverage for the second proposed domain, restricted, repetitive patterns of behavior, interests, or activities, and its four specific diagnostic criteria (i.e., stereotypies in speech, motor movements, or the use of objects, excessive adherence to routines, ritualized patterns of behavior, excessive resistance to change, highly restricted interest, hyper- or hyporeactivity to stimuli).

Results

The CASS internal scale reliability was excellent (Cronbach's α = .97; = 72). The CASS test–retest reliability (1 week) was also high (= .91; = 45). Thus, the first hypothesis was confirmed, as the CASS exhibited good internal and test–retest reliabilities.

To determine whether children in the loner group were indeed viewed by their parents as having higher levels of schizoid personality disorder traits, the Schizoid Personality Disorder scale of the CPNI was used to examine the three groups. A one-factor analysis of variance (ANOVA) revealed that the Schizoid Personality Disorder scale means for the loner group (= 66.4, SD = 16.9) and the Asperger's group (= 68.4, SD = 14.3) were significantly higher than the control group (= 47.2, SD = 10.2), F(2, 69) = 20.7, < .001 (with a large effect size, η2 = .38), although the two former groups were not statistically different from each other. In addition, 10 of the 19 children in the loner group were rated by their parents as being at least one standard deviation above the normative mean on the CPNI Schizoid Personality Disorder scale.

To test the second hypothesis regarding the ability of the CASS to differentiate among the three groups, a one-factor ANOVA was conducted upon CASS sum among the three groups, Asperger's, loners, and control, and it was significant, F(2, 69) = 67.1, < .001 (η2 = .66). A Tukey's post hoc analysis determined that the Asperger's group (= 118.8, SD = 17.2) scored significantly higher than the loner group (= 92.9, SD = 28.4), which, in turn, scored significantly higher than the control group (= 60.4, SD = 8.8). Thus, the second hypothesis was also confirmed. Additionally, to further test differences of the CASS measure upon age, a 2 (age group) × 2 (disorder group) ANOVA was conducted. Children's ages were assembled into two groups: ages 3 to 8 years and ages 9 to 16 years. The only significant effect was that of the main effect for disorder, F(2, 66) = 64.5, < .001 (η2 = .66). Thus, the children's age was not a mediating variable for these data.

To test the third hypothesis, a one-factor ANOVA was performed upon the ADHD scale of the CPNI among the three groups, and it was significant, F(2, 69) = 16.7, < .001 (with a large effect size, η2 = .33). The third hypothesis was also confirmed, as Tukey's test revealed that the Asperger's group (= 66.3, SD = 10.7) scored significantly higher on the ADHD scale than the loner group (= 55.5, SD = 15.4), which, in turn, scored significantly higher than the control group (= 45.9, SD = 11.5).

A one-factor ANOVA was also performed upon the Depression scale of the CPNI among the three groups to test the fourth hypothesis. The ANOVA was significant, F(2, 69) = 14.9, < .001 (with a large effect size, η2 = .30). Tukey's test revealed that the Asperger's group (= 64.3, SD = 11.2) and the loner group (= 55.8, SD = 15.7) scored significantly higher on the Depression scale than the control group (= 45.3, SD = 10.9). However, Tukey's test only approached significance between the Asperger's group and the loner group (= .06). Thus, the fourth hypothesis was only partially confirmed.

A one-factor ANOVA was also performed upon the Executive Function Deficits scale of the CPNI among the three groups to test the fifth hypothesis. This hypothesis was confirmed; the ANOVA was significant, F(2, 69) = 20.1, < .001 (η2 = .37). Tukey's test revealed that the Asperger's group (= 66.9, SD = 10.4) scored significantly higher than the loner group (= 58.1, SD = 15.9), which, in turn, scored significantly higher on the Executive Function Deficits scale than the control group (= 45.3, SD = 10.9).

Finally, the underlying theoretical structure of the 45-item CASS was explored through a principal components analysis (with various varimax rotations, specifying from two to five components). It appeared that a simple one-component solution appeared to best fit the data (eigenvalue = 22.1; total variance accounted for = 49.1%). It is interesting to note that Cronbach (1951) suggested that scales with a high first-factor concentration generally yield higher internal scale reliability, and this was demonstrated with the CASS, as its internal reliability with the present sample was high (α = .97).

Discussion

As hypothesized, the CASS had excellent internal scale reliability and test–retest reliability. The second hypothesis was also confirmed. ANOVA revealed that the CASS was able to differentiate significantly among the three groups, and, as predicted, the children with Asperger's scored significantly higher than the loner group, and the loner group scored significantly higher than the controls. With regard to the DSM-IV-TR contention that ADHD is often diagnosed prior to the diagnosis of Asperger's disorder, the present results also confirmed that the Asperger's group scored significantly higher on the ADHD scale of the CPNI than the loner group and the control group. Interestingly, further examination revealed that 74% (14 of 19) of the children in the Asperger's group may have met the criterion for ADHD, according to their parents' reports on the 18 items on the CPNI ADHD scale (each item represents a criterion for ADHD according to the DSM-IV-TR).

Because DSM-IV-TR also notes that Asperger's disorder can be associated with depression symptoms, depressive symptomatology was also examined among the three groups, and it was confirmed that the Asperger's group scored significantly higher on the Depression scale of the CPNI than the control group, although the Asperger's group was only marginally higher than the loner group. The last hypothesis was also confirmed: On a measure of executive function deficits, the Asperger's group scored significantly higher than the loner group, which, in turn, scored significantly higher than the control group. Finally, the 45-item CASS was examined by principal components analysis, and it was found that a simple one-component solution best fit the data. Furthermore, this first component accounted for nearly 50% of the total variance. A clear one-component solution was surprising in light of the fact that the CASS was created to provide coverage for a broad array of verbal and nonverbal behavioral dimensions, including socialization difficulties, speech and language problems, nonverbal communication difficulties, and repetitive or stereotyped behaviors. In spite of the latter groupings, all 45 items of the CASS had high intercorrelations with one another. The finding of a major one-component solution and the high intercorrelations among all of the items is also highly consistent with the inordinately high internal scale reliability (e.g., α = .97), and, as previously noted, Cronbach (1951) found that his measure of internal scale reliability would be higher in data where there was a high first-factor concentration. Of course, one limitation of the principal components analysis in this study was the small sample size, and this finding should be replicated with a much larger sample (e.g., = 450).

There is also some sense of irony in these results in regard to the popular contention that Asperger's disorder is referred to as a “mild form” of autism. The present results clearly demonstrate that the children with Asperger's had significantly higher levels of schizoid personality disorder traits, higher levels of ADHD, higher levels of depression, and higher levels of executive function deficits than controls. Thus, it appears that the basis for the popular contention must be found in the fact that individuals with Autistic disorder are so often virtually mute (e.g., approximately 50% are functionally mute in adulthood; Tincani, 2004), and so many have an associated diagnosis of mental retardation (e.g., 72% have an IQ below 70; Edelson, 2006). Therefore, individuals with Asperger's disorder have a “mild form” of autism only because so many individuals with the more severe forms of autistic disorder appear to have such devastating social and cognitive deficits.

This study was limited in the generalization of the present findings by a single parent report of not only their children's diagnosis but also of their personality and cognitive traits. Future studies may wish to include clinical interviews and clinical observations of the children to confirm their diagnosis and to include multiple ratings of the children's behavior (e.g., a second parent or guardian, teachers, caretakers, etc.). Another limitation is that this study did not take into account the different demographic characteristics of the participants (e.g., race, socioeconomic status, IQ, etc.). Thus, the generalizability of the CASS remains limited at the present time. Future studies should analyze the effects of various demographic characteristics upon the usefulness and diagnostic success of the CASS. Future research with the CASS should be conducted with samples from the more severe end of the autism spectrum.

Preliminarily, it appears that the CASS may be a useful measure to differentiate among children in the “milder” end of the autism spectrum, children who may have strong introvertive or schizoid personality traits, and children without any apparent atypical developmental behaviors. The DSM-5-proposed changes to the autism spectrum do preliminarily appear to solve the contentious issue of discriminating between Asperger's disorder and high-functioning autism and the issue of language delays as a unique characteristic of the autism spectrum but not Asperger's disorder. Thus, a single scale, such as the CASS, designed to discriminate among levels of severity in the autism spectrum appears to be worthy of further investigation.

Ancillary