Childhood Autism Rating Scale − Tokyo Version for screening pervasive developmental disorders

Authors


address: Hisateru Tachimori, Department of Mental Health Administration, National Institute of Mental Health, National Center for Neurology and Psychiatry, 1-7-3 Konodai, Ichikawa, Chiba 272-0827, Japan. Email: tachi@ncnp-k.go.jp

Abstract

Abstract To assess the utility of the Childhood Autism Rating Scale − Tokyo Version (CARS-TV), its total score was compared among 430 children with DSM-IV per subgroup (i.e. autistic disorder (AD), childhood disintegrative disorder (CDD), Asperger's disorder, and pervasive developmental disorders (PDD) not otherwise specified (PDDNOS)). Values of Cronbach's alpha were 0.91 for the PDD group and 0.89 for the non-PDD mental retardation (MR) group, and 0.93 for both groups combined. The total score was significantly higher in PDD (mean = 30.1, SD = 4.5) than in non-PDD MR (mean = 22.9, SD = 3.3), t(503) = 13.7, P< 0.0001. The cut-off to distinguish PDD from non-PDD MR was 25.5/26, with sensitivity, specificity, positive predictive value and negative predictive value of 0.86, 0.83, 0.97 and 0.50, respectively. The total score differed significantly among the four groups, with CDD and AD being significantly higher than both PDDNOS and Asperger's disorder, PDDNOS being significantly higher than Asperger's disorder and no significant difference between CDD and AD. The cut-off to distinguish AD from PDDNOS was 30/30.5, with sensitivity, specificity, positive predictive value and negative predictive value of 0.71, 0.75, 0.77 and 0.69, respectively. CARS-TV seems to be a useful instrument for differentiating between PDD and non-PDD MR and between AD and PDDNOS, although further replication is needed.

INTRODUCTION

The Childhood Autism Rating Scale (CARS) developed by Schopler et al.1,2 is one of the most widely used instruments to evaluate the degree of autism in developmentally disabled children through clinical observation by a trained rater. The CARS consists of 15 items involving definitions of autism by Rutter3 and by Ritvo and Freeman.4 Each item is rated on a four-point scale and the total CARS score (i.e. a sum of the 15 item scores) ranges from 15 to 60. The CARS has demonstrated good internal consistency,1,5,6 interrater reliability1,7 and validity.8–14 Mesibov et al.9 recommended that cut-offs of CARS for diagnosing autism be 30 in children and 27 in adolescents. However, the concept of autism used to determine the cut-off of the CARS was not as defined in DSM-IV15 or ICD-10.16

The CARS was translated into non-English languages. Kurita et al.17 developed its Japanese version, the Childhood Autism Rating Scale − Tokyo Version (CARS-TV) and demonstrated its reliability and validity, but its cut-off and sensitivity/specificity have yet to be studied. Nordin et al.18 developed a Swedish version of the CARS and demonstrated its reliability and sensitivity/specificity to diagnose DSM-III-R autistic disorder and autism spectrum disorders based on a relatively small number of patients (15 with autistic disorder, five with autism spectrum disorder and five without autism spectrum disorder (four with attention-deficit hyperactivity disorder and one with Landau–Kleffner syndrome)).

The CARS cut-off and corresponding specificity/sensitivity for autism need re-examination in light of a current diagnostic system, DSM-IV15 or ICD-10,16 based on a sufficient number of children. In addition, the CARS cut-off and sensitivity/specificity for pervasive developmental disorders (PDD) as a whole also need investigation, as they have not been reported in any version of the CARS.

Pervasive developmental disorder infants have many more difficulties in behavior and development than non-autistic mentally retarded infants of similar developmental levels and differentiation between them is not necessarily easy in clinical settings. In view of the importance of early initiation of remedial treatment for autistic infants19,20 and recent epidemiological studies reporting higher prevalence rates of autism (0.16%,21 0.17%,22 0.38%23), Asperger's syndrome (0.36%)24 and PDD (0.63%)22 than previous studies, a valid and efficient diagnostic/screening scale for PDD is important. Hopefully, such a scale could distinguish a specific PDD subtype (e.g. autistic disorder) from the other PDD subtypes.

In the present study, we attempted to clarify cut-offs and corresponding sensitivity/specificity of the CARS-TV to distinguish between PDD and mental retardation without a history of PDD and among PDD subtypes diagnosed according to DSM-IV.

METHODS

Subjects

Subjects of this study were 430 children with PDD (mean age = 80.8 months, SD = 46.4, range = 25–294; male 357, female 73) and 75 children with mental retardation (MR) and without a history of PDD (non-PDD MR) (mean age = 80.5 months, SD = 50.1, range = 37–352; male 26, female 49) diagnosed according to DSM-IV, having consecutively visited one of three clinics specializing in developmental disorders, the Nerima Welfare Center for Handicapped Persons, the Child Guidance Clinic affiliated with the National Welfare Foundation for Disabled Children and the Kawasaki Central District Center for Remedial Therapy for Handicapped Children for the last 10 years. The mean age did not differ significantly between PDD and non-PDD MR groups. The extreme difference in the number of children between the two groups possibly resulted from the fact that the three facilities were well known for specializing in autistic children and, therefore, non-autistic children were less likely to be brought to the facilities for their milder development and adaptation difficulties than autistic children.

In each of the three facilities, a clinical team consisting of child psychiatrists, child psychologists, pediatric neurologists, speech pathologists, registered nurses and social workers evaluated children and arranged their treatments, including enrolment in affiliated day-care facilities, parental counseling and follow-up observations in basically the same manner. The clinical teams in the three facilities had used relevant DSM schemes to make a diagnosis of visiting children since the 1980s. They re-diagnosed, by consensus, all subjects for the present study according to DSM-IV by using all the available clinical data on the children other than the CARS-TV data, including detailed clinical records and questionnaires on development and symptoms that were filled out by parents.

The 430 PDD children were classified into four DSM-IV diagnostic groups: 212 (180 male, 32 female) with autistic disorder, 31 (24 male, seven female) with Asperger's disorder, six (five male, one female) with childhood disintegrative disorder (CDD) and 181 (148 male, 33 female) with pervasive developmental disorder not otherwise specified (PDDNOS). There was no case of Rett's disorder in the 430 PDD children. In fact, none of the 75 non-PDD MR children had ever met the PDD definition or diagnostic criteria for any PDD subtype in DSM-IV.

In 290 of the 505 children (430 PDD and 75 non-PDD MR), IQ was measured on the Japanese version of the Stanford-Binet Intelligence Scale at the time of first visit by experienced psychologists. In the other 215 children in whom an IQ was not measurable, a developmental quotient (DQ) measured by the Infant Mental Development Questionnaire or the New Kyoto Developmental Scale was used for an IQ, since such DQ were well correlated with IQ.25 The mean IQ/DQ (SD) of the subjects was 59.0 (26.6) in the PDD group and was 62.6 (16.4) in non-PDD MR group without a significant difference between them. The mean IQ/DQ (SD) also did not differ significantly by sex in the two groups: 58.6 (26.8) in males and 60.7 (25.8) in females in the PDD group, t(428) = 0.609, P = 0.54; and 65.0 (16.9) in males and 58.0 (14.9) in females in the non-PDD MR group, t(73) = 1.78, P = 0.079.

In the 505 children, the number (%) of children with mental retardation (IQ < 70), borderline intellectual functioning (70 ≤ IQ < 85) and normal intelligence (IQ ≥ 85) were 339 (67.1), 68 (13.5), and 98 (19.4), respectively, and did not differ significantly with corresponding numbers (%) in 396 IQ-measured children in the CARS initial study by Schopler et al.,1 279 (70.5), 73 (18.4) and 44 (11.1), respectively.

In the three facilities, experienced psychologists rated children on the CARS-TV before the examination of the children by child psychiatrists without information about the children's DSM-IV diagnoses.

The present study using CARS-TV, intelligence/developmental tests and demographic data routinely collected for clinical services for developmentally disabled children in the three facilities was approved by administrators of the three facilities and was conducted through cooperation of staff members of each facility.

Method

The method was the same as the original CARS. The CARS-TV consists of 15 items (i.e. relationships with people, imitation, affect, use of body, relation to non-human object, adaptation to environmental change, visual responsiveness, auditory responsiveness, near receptor responsiveness, anxiety reaction, verbal communication, non-verbal communication, activity level, intellectual functioning and general impressions) rated on a four-point-point scale (1, behavior appropriate for age; 2, mildly abnormal; 3, moderately abnormal; and 4, severely abnormal) with three midpoints (i.e. 1.5, 2.5 and 3.5) being used if needed to provide a total score (i.e. a sum of the 15 item scores) ranging from 15 to 60. (A copy of the CARS-TV is available from the last author (H. Kurita) upon request.)

We analyzed the data in three steps. First, we calculated a Cronbach's alpha for the PDD group, the non-PDD MR group and the entire group of 505 children to examine internal consistence reliability of the CARS-TV in the present study. Second, we compared the total CARS-TV score between PDD and non-PDD MR groups by t-test. Then, we searched an appropriate cut-off of CARS-TV total score to distinguish PDD from non-PDD MR by examining sensitivity and specificity for each cut-off. Third, we compared the total CARS-TV score among the four PDD subtypes by one-way analysis of variance with post-hoc Tukey–Kramer test. Then, we studied the appropriate cut-off and sensitivity/specificity of the CARS-TV for distinguishing one PDD subtype from another (e.g. autistic disorder vs PDDNOS; autistic disorder vs Asperger's disorder; and Asperger's disorder vs PDDNOS).

A significant level was set at P < 0.05 (two-tailed). All statistical analyses were performed with the StatView package for Macintosh, version 5.0 (SAS Institute Inc. Cary, NC, USA).

RESULTS

Internal consistency

Values of Cronbach's alpha as a measure of internal consistency reliability were 0.91 for the PDD group and 0.89 for the non-PDD MR group, and 0.93 for both groups combined.

PDD versus non-PDD mental retardation

The total CARS-TV score was significantly higher in PDD (mean = 30.1, SD = 4.5) than in non-PDD MR (mean = 22.9, SD = 3.3), t(503) = 13.7, P < 0.0001. The total CARS-TV score and ages showed a significant but weak correlation in PDD (r = −0.12, P = 0.010) and no significant correlation in non-PDD MR, and PDD and non-PDD MR combined.

Table 1 shows that the best cut-off to distinguish PDD from non-PDD MR was 25.5/26 with sensitivity of 0.86 and specificity of 0.83. At this cut-off, a positive predictive value was high, 0.97, but a negative predictive value was low, 0.50.

Table 1.   Sensitivity, specificity, and positive and negative predictive values for CARS-TV cut-offs for screening PDD against mental retardation without history of PDD
Cut-offSensitivitySpecificityPositive
predictive
value
Negative
predictive
value
  1. The most appropriate cut-off and its related measures are indicated in bold-faced italic. Numbers of children were 430 and 75 in groups of PDD and of mental retardation without history of PDD, respectively.

  2. CARS-TV, Childhood Autism Rating Scale − Tokyo Version; PDD, pervasive developmental disorders.

23.0/23.50.930.570.930.61
23.5/24.00.910.640.940.56
24.0/24.50.900.710.950.55
24.5/25.00.890.730.950.53
25.0/25.50.870.790.960.51
25.5/26.00.860.830.970.50
26.0/26.50.820.840.970.45
26.5/27.00.790.870.970.42
27.0/27.50.760.880.970.39
27.5/28.00.710.910.980.35

Comparison among PDD subtypes

Table 2 shows that the mean total CARS-TV score differed significantly among the four groups. Post-hoc tests showed that it was significantly higher in CDD and autistic disorder than both PDDNOS and Asperger's disorder, and in PDDNOS than Asperger's disorder, but did not differ significantly between CDD and autistic disorder.

Table 2.   Total CARS-TV scores in four PDD subtypes
PDD subtype (n)Total CARS-TV score
MeanSDRange
  • F3,426  = 52.1, P  < 0.001.

  • a,b,c

    Means with a different superscript differ significantly from each other (Tukey–Kramer test, P  < 0.05).

  • CARS-TV, Childhood Autism Rating Scale − Tokyo Version; CDD, childhood disintegrative disorder; PDDNOS, pervasive developmental disorder not otherwise specified.

CDD (6)33.6a3.429.5–38.5
Autistic disorder (212)32.3a4.418.0–51.0
PDDNOS (181)28.2b3.318.5–37.5
Asperger's disorder (31)25.6c3.618.0–34.0

Cut-offs for PDD subtypes

Table 3 shows that the best cut-off to distinguish autistic disorder from PDDNOS was 30/30.5 with a sensitivity of 0.71 and specificity of 0.75. At this cut-off, positive and negative predictive values were 0.77 and 0.69, respectively.

Table 3.   Sensitivity, specificity, and positive and negative predictive values for CARS-TV cut-offs for screening autistic disorder against PDDNOS
Cut-offSensitivitySpecificityPositive
predictive
value
Negative
predictive
value
  1. The best cut-off and its related measures are indicated in bold-faced italic. Numbers of children were 212 and 181 in autistic disorder and PDDNOS groups, respectively.

  2. CARS-TV, Childhood Autism Rating Scale − Tokyo Version; PDDNOS, pervasive developmental disorder not otherwise specified.

27.0/27.50.910.350.620.77
27.5/28.00.890.430.650.77
28.0/28.50.860.500.670.75
28.5/29.00.830.560.690.73
29.0/29.50.790.630.710.72
29.5/30.00.740.690.730.69
30.0/30.50.710.750.770.69
30.5/31.00.670.800.790.67
31.0/31.50.610.820.800.64
31.5/32.00.540.860.820.62

We also examined cut-off, sensitivity, specificity and positive and negative predictive values between autistic disorder and milder autistic conditions (i.e. PDDNOS and Asperger's disorder combined) and between Asperger's disorder and PDDNOS, but did not examine them in CDD, because the number of CDD was small and was not significantly different in the total CARS-TV score from autistic disorder. The cut-off to distinguish autistic disorder from the milder autistic conditions (i.e. PDDNOS and Asperger's disorder) was 29.5/30.0, with sensitivity, specificity, positive predictive value and negative predictive value, 0.74, 0.71, 0.72 and 0.73, respectively. The cut-off for distinguishing PDDNOS from Asperger's disorder was 27/27.5, with sensitivity, specificity, positive predictive value and negative predictive value of 0.65, 0.68, 0.92 and 0.25, respectively.

DISCUSSION

The present study showed that the CARS-TV cut-off (30.0/30.5) for autistic disorder was basically the same as that (30) of the CARS for autism in American and European studies, suggesting an aspect of cross-validity of the CARS-TV. This and its satisfactory internal consistency shown in the present study together with previous findings17 on its reliability and validity would confirm the utility of the CARS-TV as a screening and diagnostic scale for autism.

Although the study by Schopler et al. of 271 autistic children seem comparable to our 212 children with autistic disorder, despite the difference in diagnostic systems employed, the identicality of the cut-off between the study of Schopler et al. and ours is uncertain, as no American study on specificity/sensitivity of the CARS based on DSM-IV or ICD-10 exists. This would best be addressed by a study to compare CARS ratings by American professionals and CARS-TV ratings by Japanese counterparts of the same autistic and non-autistic children, diagnosed according to DSM-IV, through direct or video-taped observations.

The present study was the first to report a CARS cut-off (25.5/26) to discriminate PDD from non-PDD MR. Although a CARS cut-off for PDD has never been reported in American and European studies, the magnitude of the CARS-TV cut-off for PDD seems reasonable in view of the constitution of our PDD subgroups. Of the 218 children with PDD other than autistic disorder, 181 (83.0%), 31 (14.2%), and six (2.8%) were diagnosed as PDDNOS, Asperger's disorder and CDD, respectively. Asperger's syndrome is the highest functioning PDD and has milder autistic symptomatology than autistic disorder. PDDNOS is also a milder variant of PDD than autistic disorder.26 In addition, the number of children with CDD was small and not significantly different from those with autistic disorder in the total CARS-TV score. In the present study, as children with PDD other than autistic disorder (CDD, PDDNOS and Asperger's disorder) had lower total CARS-TV scores, it is quite reasonable that the cut-off for PDD, in general, was lower than that for autistic disorder. Although the total CARS-TV score seems unable to distinguish autistic disorder from CDD, to obtain a history of normal development at least for the first 2 years of life would certainly help clinicians discriminate CDD from autistic disorder.

The small negative predictive value for PDD (0.50) is a problem. In other words, we cannot use the CARS-TV cut-off to exclude the possibility of PDD, as a false-negative rate (1 − negative predictive value = 0.50) also equal to a chance rate. This might have resulted from contrasting the small number of non-PDD patients to the great number of PDD patients recruited from the facilities specialized for PDD. A further study employing a large number of non-PDD patients is needed to address this problem. For the time being, however, the CARS-TV cut-off for PDD (25.5/26) should be used to predict the presence of PDD but not the absence of it in clinical settings. The cut-off for PDD itself also needs replication in different samples of PDD children including those from the USA and European countries.

Although Mesibov et al.9 recommended a CARS cut-off of 27 instead of 30 when used to assess adolescents with autism, our sample did not support their recommendation; that is, the total CARS-TV score was not correlated with age (i.e. decreased over time) in PDD and non-PDD MR combined in accordance with the finding of Garfin et al. that CARS ratings were stable into adolescence.5 Whether or not CARS ratings change over time remains an open question. To address this problem, a longitudinal study on CARS evaluation of PDD patients is needed.

The present study also showed that the CARS-TV could differentiate autistic disorder from PDDNOS at a cut-off of 30.5/31 with specificity and sensitivity slightly lower than those for the cut-off to discriminate PDD as a whole from non-PDD mental retardation. This property of the CARS-TV seems clinically important, given the difficulty of differentiating PDDNOS from autistic disorder in clinical settings and a larger prevalence rate in PDDNOS than autistic disorder.22 We failed to identify a useful cut-off for Asperger's disorder. As total CARS-TV scores in almost all of the children with Asperger's disorder in the present study were in the non-autistic range as they were in non-PDD MR children, CARS-TV seems quite limited in its ability to separate Asperger's disorder from non-PDD MR and PDDNOS. A specialized scale is needed to screen Asperger's disorder and other high-functioning PDD.

Although the CARS-TV assessment is mainly based on the direct observation of children's behavior, it may partly depend on information from their parents. Thus, it seems possible that CARS-TV scores are sometimes influenced by parents’ mental state, educational and socioeconomic level, and relationship with their children. It is important that clinicians should know this limitation and make a judgment synthetically after considering all information.

Although there are several tasks left ahead and limitations, this study, based on a large number of PDD children, demonstrated the usefulness of the CARS-TV to screen/diagnose PDD and autistic disorder. Given recent epidemiological studies reporting high prevalence rates of autism,21–23 Asperger's syndrome,24 and PDD22 and growing recognition of the importance of early intervention for PDD infants,26 the CARS-TV is a useful instrument in the field of developmental disorders.

CONCLUSION

The CARS-TV differentiated PDD from non-PDD MR at a cut-off of 25.5/26 and differentiated autistic disorder from PDDNOS at a cut-off of 30/30.5. Despite some limitations, CARS-TV is a useful instrument to screen/diagnose PDD and autistic disorder and to supplement DSM-IV or ICD-10 diagnosis.

ACKNOWLEDGMENTS

This study was supported, in part, by the Research Grant (11A-6-14) for Nervous and Mental Disorders from the Ministry of Health, Labor and Welfare, Japan. We thank Ms Tomoko Nakano at the Nerima Welfare Center for Handicapped Persons, Ms Yoko Hayashi and Keiko Shimoyamada, RN at the Child Guidance Clinic affiliated with the National Welfare Foundation for Disabled Children, and Mr Hiromi Ishida at the Kawasaki Central District Center for Remedial Therapy for Handicapped Children for their assistance in data collection.

Ancillary