The Cochrane Library and Autism Spectrum Disorder: An Overview of Reviews

Authors


Editors' note:Umbrella reviews, compiling evidence from multiple Cochrane reviews into one accessible and usable document, are a regular feature of this journal. Our aim for each umbrella review is to focus on the treatment question, ‘which treatment should I use for this condition?’. It is our hope that the umbrella review will serve as a ‘friendly front end’ to The Cochrane Library, allowing the reader a quick overview (and an exhaustive list) of Cochrane reviews relevant to the clinical decision at hand.

Background

Description of the condition

Autism Spectrum Disorders (ASD) are life-long neurodevelopmental disorders characterized by impairment in reciprocal social interactions, impairment in verbal and non-verbal communication skills, and restricted repetitive and stereotyped patterns of behaviour, interests and activities (American Psychiatric Association, 2000). The spectrum includes conditions such as Autistic Disorder (AD), Asperger syndrome, Atypical Autism, and Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS). As well, many individuals affected by autism have additional diagnoses, most often intellectual impairment but also other physical, behavioural and emotional problems. Current practice requires that autism and related disorders are diagnosed using either the Diagnostic and Statistical Manual of Mental Disorders (DSM)1 or the International Classification of Diseases (ICD)2 classification systems.

The reported prevalence of ASD has increased in the past 15 to 20 years. Possible explanations are that the incidence of ASD has increased or, alternatively, that the increase represents changes in diagnostic tools, service availability and awareness of ASD among health professionals and the general public. Current estimations of prevalence of Autistic Disorder vary between 10–15/10,000 children3, but possibly greater than 20 of every 10,000 children have dysfunction which warrants diagnosis at any point along the autism spectrum4. Some recent studies have reported rates as high as 1 in 150 children3. ASD affects boys approximately four times more frequently than girls5.

The prognosis for children with ASD varies and is determined by factors such language development, intelligence, social adjustment and the ability to function independently. Individuals with ASD frequently pose considerable behavioural challenges to their caregivers6, 7. Parents of children with autism report higher levels of stress and mental ill health than parents of children with other developmental disorders6. In adulthood, autism greatly restricts vocational and recreational opportunities, adding considerably to the costs of care for families and community.

Although symptoms often persist through adulthood, timely detection and treatment of ASD may be an important factor in improving outcomes8. Over the past 20 years, a variety of treatment approaches have been developed in an attempt to remediate the deficits associated with ASD. Interventions vary extensively in terms of their underlying theoretical framework, mode of delivery, intensity, degree of parental involvement and comprehensiveness. Current treatments include pharmacological therapies, behavioural and educational therapies, sound therapy, various complementary therapies9 and therapies well accepted for use in developmental problems, such as speech and language and occupational therapy.

Description of the interventions

Drug-based treatments

A variety of drugs are used to treat ASD. Some of these target specific symptoms such as hyperactivity. Others have been used following reports of improvements in autistic symptoms when a drug was administered for another purpose. Drug-based treatments are most frequently used in addition to behavioural treatments, rather than as a stand-alone therapy10.

Risperidone

Antipsychotic drugs are the most frequently prescribed psychoactive agents used for treatment of autism11; of these, risperidone is the most widely used atypical antipsychotic. It is prescribed to reduce specific autistic symptoms and behaviours, and it has been suggested that risperidone might improve language use and reduce dysfunctional behaviour. It is also used as an adjunct to behavioural treatments. There is a concern that the use of risperidone may cause weight gain, physiological and/or behavioural adverse effects, especially when used in children10.

Selective serotonin re-uptake inhibitors

Serotonin is linked to the mediation of several psychological processes, many of which are altered in autism, including mood, social interaction, sleep, obsessive–compulsive behaviours and aggression12. Selective serotonin-reuptake inhibitors (SSRIs) have been used to treat some of these disorders when they occur in isolation. It has therefore been thought possible that SSRIs will result in improvement of similar autism symptoms. Of concern are possible adverse effects that include increases in maladaptive behaviours, urinary retention and seizures13.

Secretin

Secretin is a gastro-intestinal hormone produced in the intestine. Secretin started to be used as a treatment for autism after a report that the autistic symptoms of three children improved following the administration of secretin during diagnostic testing for gastrointestinal disorders14. The exact role of secretin and of its mechanism of action in the central nervous system has not been determined. There are currently no reports of serious adverse effects.

Behavioural and educational treatments

There are many behavioural and educational treatments used for autism. Based on learning theory, behavioural interventions are characterized by the use of instrumental learning techniques. Behavioural methodology has been evolving over time and new techniques have been introduced. However, applied behaviour analysis and discrete trial learning still remain the core features of behavioural intervention in ASD15. Under this paradigm, behaviours are broken into discrete components and then behavioural techniques of reinforcement, shaping, chaining, and prompting procedures are used. Many of these treatments are well regarded and have been used, in varying formats and with modifications to content, over many years. A recent systematic review of 87 trials of behavioural and educational interventions concluded that, in spite of the many published studies on the subject, there is great variation in the reports of the effectiveness of these interventions and therefore, there is no definitive answer regarding the ‘best’ approach to treat the diagnostic features and other manifestations of autism16.

Behavioural and educational interventions also vary in relation to the timing of their delivery, site of delivery and person administering the therapy. There is considerable variation between programs as to who delivers them (some are delivered by professionals, others by parents) and where they take place (some take place in school, community or clinical settings, while others are home-based). Furthermore, some are delivered to individuals whereas others are designed for group settings.

Although early intervention programmes may differ in content, their proponents all advocate treatment implementation when the child is as young as possible. Early intervention using behavioural and educational treatments can exist in a variety of forms17 including applied behaviour analysis or others which have an educational framework, such as project TEACCH (Treatment and Education of Autistic and related Communication handicapped Children)18. A number of early intervention programmes emphasize creation of naturalistic communication opportunities, enhancing motivation for social interaction and prompting specific social behaviours (e.g. 7, 19, 20).

Parent mediated early intervention

The involvement of parents in implementing intervention strategies to help their autistic children has been reported for more than 30 years18. The potential benefits may include increased skills and reduced stress for parents as well as children. Training parents as ‘therapists’ allows intervention to begin early, to involve consistent handling, and ensures that intervention is appropriate in enhancing children's earliest social relationships. However, there are possible adverse effects including the impact of therapy on family function and reduced access to outside services.

Sound therapies

Sound therapies have arisen because of two main observations. One observation is that some people with ASD are sensitive to sound. The second is that the communication skills in children with autism improved when exposed to music. There is a perception of little likelihood of harm resulting from sound therapy.

Music therapy

Advocates of music therapy refer to ‘musical qualities’ seen in the dialogues between mothers and infants and the sensitivity of young children to rhythm and melody. Conversation is thus considered to be a musical entity and music is seen as an effective medium to engage non-verbal social exchange for individuals with ASD. Music therapy has been used as a treatment for autism following anecdotal evidence of improvements in communication skills in children with autism21 when they are exposed to music. Music therapy is usually provided as individual therapy and may include listening to music, singing and improvisation. Depending on the abilities of the participant, reflection on any associations brought up by the music may be included22. There are no published reports of serious adverse effects.

Auditory Integration Therapy

Auditory integration training (AIT) was developed in response to the theory that abnormal sensitivity to certain frequencies of sound waves was associated with a range of behaviour and learning problems23. It was postulated that AIT would ‘re-educate’ the hearing process. Following anecdotal reports of success, AIT was used as a treatment. AIT involves 10 hours of listening to electronically modified music delivered by headphones during two half-hour sessions over 10 days. Music is modified by removal of peak frequencies to which the individual is considered hypersensitive and sounds are modulated by random dampening of high and low frequencies and intensities. Variation exists, including the Tomatis method, which includes modified human voice and music, and Samonas Sound Therapy, which includes filtered music, voice and nature sounds. No serious adverse effects have been reported, but there are potential concerns about hearing loss from headphone use if music is too loud. The potential for parental frustration when trying to engage children in the activity also exists, as does the potential for financial stress given that the treatment is offered only by private practitioners.

Complementary therapies

A variety of complementary therapies are used for the treatment of ASD. A survey of parents in the United States reported 95% of respondent parents used some form of complementary therapy24. These include various dietary interventions, mega-doses of vitamins, anti-fungal treatment, probiotics, digestive enzymes, chiropractic manipulation and heavy metal chelation.

Gluten-/casein-free diet

The possibility that diet exacerbates diagnostic and associated features of autism has been postulated since early studies of vitamin therapy in the 1950s. In the early 1980s, attention was drawn to animal studies of the effects of opioids and the similarity of behaviours of opioid-affected animals to autistic traits in children25. Further research found elevated levels of urinary peptides in some children with autism. It has been proposed that these urinary peptides are the result of incomplete protein breakdown. Subsequently, there have been several reports of a variety of gastrointestinal symptoms in autistic children including abdominal pain, diarrhoea, and bloating. The similarity of identified urinary peptides to the breakdown products of the milk protein, casein, and gluten from cereals has fuelled supposition that dietary exclusion of these products will decrease autistic behaviours26. The adoption of a gluten- and casein-free diet is now a widely used treatment for autism. Parents get much of their information and support from on-line or non-mainstream medical sources, although some practitioners recommend the treatment. There are concerns that the use of a restrictive diet in children with already fussy eating behaviours may risk nutritional deficiencies. Further, financial and organisational demands on families are increased with the use of the diet, given that extra food must be purchased and prepared. The many variations in recommendations for additional ‘gut support’ treatments has led to confusion among parents, as well as additional expense.

Combined vitamin B6 and magnesium

Reports of improvements in autistic symptoms following treatment with pyridoxine (vitamin B6) first appeared in 196827. However, large doses of vitamin B6 resulted in adverse events including irritability, hypersensitivity to sounds, and enuresis. It was found that these adverse events could be prevented by the administration of magnesium and over the next 20 years, a number of non-randomized, open-labelled studies were published that indicated that vitamin B6 and magnesium could improve communication, interpersonal skills and physiological function. The use of blinded, placebo-controlled RCTs in the 1990s began to provide contradictory results28, 29. There is no likely biological mechanism for this treatment.

Why It is Important to do this Overview

Autism is a complex, lifelong disorder that places great burdens on affected individuals and their families. There is no known cause and treatments are many and varied, often with little evidence to recommend them. Because families are keen for any improvement, they are often willing to try suggested therapies that are not proven and incur a cost in time, money and/or potential adverse events, for the family and the child. This overview of current evidence from systematic reviews published in the Cochrane Library for specific treatments for ASD is required to describe which treatments may be effective or show promise and require further evaluation, and those that are unlikely to help most children with autism.

The purpose of these overviews of reviews is to serve as a ‘friendly front end’ to the Cochrane Library, offering the reader a quick overview (and exhaustive list) of the Cochrane reviews relevant to a particular condition. There is limited evidence currently available within the Cochrane Library, and hence within the scope of this overview, about the effectiveness of behavioural and educational interventions, which are a major part of current autism treatment. As such, this overview cannot be a comprehensive review of all autism interventions, and we are not able to compare the effectiveness of all currently used treatments for autism spectrum disorders. However, this overview does shed light on many treatments currently being used and on the methods that are being used to provide much-needed evidence. The findings will be useful for families, health professionals, policy makers and researchers.

Objectives

To summarize Cochrane reviews which evaluate the effectiveness of interventions for ASD.

Methods

Criteria for considering reviews for inclusion

Any reviews currently published on the Cochrane Library or reviews ready for publication as a first or updated review, that evaluate interventions for ASD.

Search methods for identification of reviews

The Cochrane Database of Systematic Reviews was searched for all systematic reviews of any intervention for ASD. The term ‘autism’ was entered and restricted to record title, abstract or keyword.

Data collection and analysis

Selection of reviews

Reviews were selected by one author (DW) by assessing methods of reviews found in the search of The Cochrane Database of Systematic Reviews. Seven reviews and one protocol3 were found; the protocol was excluded and all seven reviews were included. Table 1 details the characteristics of included reviews.

Table I. Characteristic of included reviews
TitleAuthorsDate of most recent substantive amendmentNumber of studiesStudy sample size rangePopulationInterventionComparisonOutcomes
Intravenous secretin for autism spectrum disorderWilliams KJ, Wray JJ, Wheeler DMMay 2005146–952–18 yrsIntravenous secretin at any dose or frequencyPlacebo25 outcome measures, including diagnostic features, communication, behaviour, visio-spatial skills, affect and adverse events
Risperidone for autism spectrum disorderJessner OS, Aref-Adib M, Coren EOctober 2006331–1015–43 yrsRisperidone at any dose or frequency by any route of administrationPlaceboClinical Global Impression Scale, Yale-Brown Obsessive Compulsive Scale, Ritvo-Freeman Real Life rating Scale, Aberrant Behavior Checklist, Nisonger Child Behavior Scales, Vineland Adaptive Behavior Scales, Self-injurious Behaviour Questionnaire, Visual Analog Scale (clinician and parent) Parent-derived Target Symptoms Scale
Music therapy for autistic spectrum disorderGold C, Wigram T, Elefant CJanuary 200634–102–9 yrsActive or passive music therapyPlaceboCommunicative skills, gestural; communicative skills, verbal; behavioural problems
Auditory integration training and other sound therapies for autism spectrum disordersSinha Y, Silove N, Wheeler D, Williams KSeptember 2003610–803–39 yrsAuditory integration training, 2 sessions per day for 10 daysPlaceboAberrant Behavior Checklist, Autism Behaviour Checklist Fisher's Auditory Problem Checklist, Connors Rating Scale, Leiter International Performance Scale, Reynell Language Development Scales-III, Hearing Sensitivity Questionnaire, Pure Tone Discomfort Test, Sound Sensitivity Questionnaire, direct observation
Combined vitamin B6-magnesium treatment in autism spectrum disorderNye C, Brice AAugust 2005312–53–18 yrsCombined oral Vitamin B with magnesium, min 1 wk, max 52 wksPlacebo or no treatmentVerbal behaviour, non-verbal behaviour, social interaction
Gluten and casein free diet for autism spectrum disorderMillward C, Ferriter M, Calver S, Connell-Jones GPublished 2004215–202–16 yrsDiets which eliminated gluten and/or caseinNormal dietUrinary peptides, behavioural assessment, communication and linguistic ability, cognitive function, motor ability, communication, social isolation, in-home observations
  Unpublished update 2007      
Parent mediated early intervention for autism spectrum disordersDiggle T, McConachie H RPublished 2002424–3518–72 monthsInterventions with parents as mediators Individual or groupStandard care/special educationPrimary outcomes included: child language progress, child positive behavioural change
  Unpublished update 2007     Secondary outcomes included: parent-child interaction style, parent confidence, reduction in levels of parental stress

For two of the included reviews, parent-mediated early intervention for young children with ASD and gluten- and casein-free diets for ASD, we used unpublished updates provided by The Cochrane Developmental Psychosocial and Learning Problems Group31, 32. The parent-mediated early intervention review is still undergoing editorial revision. However, the findings reported in this overview are not anticipated to change in the published version. This update will include two additional trials that are not in the currently published version33, 34. The gluten- and casein-free diet review has been submitted for publication and will appear in Issue 2, 2008, of The Cochrane Library. It incorporates one new trial, and a report of supplementary data from a study that was already included in the currently published version35, 36.

Data extraction and management

Data were extracted by one reviewer (DW) and entered directly into electronic tables. Data were subsequently verified by a second reviewer (JD) and any discrepancies resolved by consensus. Additional or missing data from trials were not sought as review authors had tried to obtain any such data at the time of review publication.

Assessment of methodological quality of included reviews

All of the seven systematic reviews included in this overview are published in The Cochrane Library. As such, they have had peer-review throughout their process of development and prior to publication. To assess the quality of included systematic reviews, each review was appraised according to the Systematic Review Appraisal Sheet of the Centre for Evidence-based Medicine, University of Oxford (http://www.cebm.net /index.aspx?o = 1157). Results of these assessments are shown in Table 2. The assessment was carried out independently by DW and JS and differences were resolved by consensus.

Table II. Systematic review appraisal
 Was the question addressed by the review clearly stated?Is it unlikely that important, relevant studies were missed?Were the criteria used to select articles for inclusion appropriate?Were included studies sufficiently valid for the type of question asked?Did the review authors estimate if differences between studies were significant and explore possible reasons for heterogeneity?Were the results presented in a clear and useful manner?
Intravenous secretin for autism spectrum disorder40YesYesYesYesYesYes
Risperidone for autism spectrum disorder10YesYesYesYesYesYes
Music therapy for autistic spectrum disorder42YesYesYesYesYesYes
Auditory integration training and other sound therapies for autism spectrum disorders41YesYesYesYesYesYes
Combined vitamin B6-magnesium treatment in autism spectrum disorder38YesYesYesYesYesYes
Gluten and casein free diet for autism spectrum disorder32YesYesYesYesYesYes
Parent mediated early intervention for autism spectrum disorders31YesYesYesYesYesYes

All included reviews have a clearly stated question about therapy, all have included comprehensive searches including both MESH terms and text words, and all have stated methodological and clinical inclusion and exclusion criteria a priori. The included studies were all randomized controlled trials and were therefore all valid given the type of question asked. All reviews presented results in a clear and useful manner, although meta-analysis was not always possible as discussed in detail later. All of the reviews either included studies whose results were similar, or explored possible reasons for heterogeneity where the results across studies were not similar. Inclusion of studies whose results were not similar does not directly reflect lower quality of the reviews themselves. However, it must be noted that three reviews (secretin, risperidone, AIT) reported high heterogeneity at meta-analysis, and therefore any evidence obtained from these meta-analyses should be considered of lower quality.

All studies included an assessment of the quality of each included trial. The included reviews were published before the release of RevMan 5, therefore, all reviews but one31 have used the Cochrane criteria to assess included trial quality, in use for RevMan 4.2 and earlier. That is:

  • a)indicates adequate concealment of the allocation (e.g. by telephone randomisation, or use of consecutively numbered, sealed, opaque envelopes);
  • b)indicates uncertainty about whether the allocation was adequately concealed (e.g. where the method of concealment is not known);
  • c)indicates that the allocation was definitely not adequately concealed (e.g. open random number lists or quasi-randomisation such as alternate days, odd/even date of birth, or hospital number).

Blinding of trials included in reviews was also assessed by review authors for whether both participants and outcome assessors were blind to the allocation status of participants. For this umbrella review, ratings were recorded as ‘met’, ‘unmet’ or ‘unclear’ criteria as described in The Cochrane Handbook37.

Trial quality was further assessed based on percentage loss to follow-up in all systematic reviews and on intention-to-treat analysis in all the reviews except one (Vitamin B6 & Mg). Standardization of outcome assessment was assessed in three reviews (Secretin, AIT, Parent-mediated). The Vitamin B6 & Mg review excluded studies that reported more than 20% attrition or with inadequate allocation concealment.

The parent mediated early intervention review described trial quality in terms of a risk of bias, in line with RevMan 5. While the language may differ, the categories are identical to the above rankings. Rankings used were as follows:

  • A—Low risk of bias (adequate allocation concealment)

  • B—Moderate risk of bias (questionable allocation concealment)

  • C—High risk of bias (inadequate allocation concealment)

However, the review did exclude studies from meta-analysis if they had a high risk of bias (category C).

All studies used statistical tests of heterogeneity to explore if included trials presenting similar outcomes were homogeneous. The levels of heterogeneity were assessed using I2 (39) with results ranging from 0–62%. Subgroup and sensitivity analyses were not possible in any of the included reviews, largely due to small numbers of studies for any one outcome and small sample sizes of included studies, so potential sources of heterogeneity and clinical and methodological diversity could not be assessed.

Results in this overview include only synthesized estimates of effect of two or more studies which were presented using a forest plot in included systematic reviews.

Data reported in this overview

Continuous and dichotomous data were presented as reported in the included reviews. Where meta-analyses were reported, standardized mean difference (SMD), weighted mean difference (WMD) or relative risk (RR), where applicable, were taken from published analyses. All meta-analyses were reported as published. Forest plots containing only one study were not reported. Continuous data were not expressed as dichotomous data. Trial data that could not be synthesized in the original systematic reviews due to paucity of data, heterogeneity, or the use of multiple, unrelated outcomes measures, were not presented in this overview. Therefore, for individual trials included in published systematic reviews but not included in meta-analyses, we refer readers back to the systematic review or the primary studies.

No indirect comparisons of interventions were calculated or mixed treatment meta-analysis performed. Data contained in the review were not considered sufficiently robust to warrant any further analysis.

Results

Description of included reviews

The seven included reviews evaluated a variety of interventions for ASD. Details of the participants, interventions and outcomes are summarized in Table 1.

Methodological quality of included trials

All included reviews reported problems with methodological quality of trials, presentation of results and the use of a variety of outcome measures. Meta-analyses were limited in all cases.

Allocation concealment

The adequacy of allocation concealment for trials included in the systematic reviews is presented in Table 3.

Table III. Number of studies for categories of allocation concealment and with ‘adequate’ blinding of outcome assessors for the studies included in each systematic review
 Number of trialsAllocation concealmentBlinded outcome assessors
AdequateInadequateUnclearAdequate
Secretin40146177
Auditory Integration Training4160606
Vitamin B6 and Mg3830033
Risperidone1030033
Music therapy4230032
Parent-mediated early intervention3144004
Gluten and casein free diet3222002

Only the parent-mediated early intervention and gluten and casein free diet reviews reported adequate allocation concealment in all included studies. The other reviews reported poor attention to allocation concealment by trialists.

Blinding of outcome assessors

Accurate assessments of blinding of outcome assessors was hard to ascertain in some reviews, largely due to trialists reporting ‘double blinding’ without reporting further details about whether this phrase referred to researchers, participants, parents or outcome assessors. Table 3 shows the number of trials where trials were said to be either ‘triple blind’ or have ‘blinding’ of outcome assessors. Blinding in the other studies was either unknown or inadequate.

Loss to follow-up

Most trials included in reviews reported losses of less than 20%. Two of three studies included in the risperidone review reported attrition rates of 20–25%. Two trials in the AIT review did not report attrition data. In all reviews, paucity of data suitable for inclusion in meta-analysis precluded a sensitivity analysis to estimate the possible effects of loss to follow-up.

Cross-over trials

Cross-over trials were common and were included in all reviews except the Risperidone and Parent-mediated early intervention reviews. In all reviews where cross-over design was used, reviewers expressed concerns with the adequacy of ‘wash-out’ periods. Two of three trials in the Magnesium vitamin B6 review were cross-overs, one of six AIT trials, two of three music therapy trials, nine of 14 secretin trials, and one of two gluten- and casein-free diet trials were cross-over design. The authors of the secretin systematic review used first phase data only because of concerns with ‘wash-out’. Analysis on this basis was consistent with Cochrane statistical recommendations at that time.

Data published in a form not suitable for inclusion in meta-analysis

All reviews reported presentation of inadequate data in some trials for use in meta-analysis, such as presentation of estimated p-values with no standard deviation. In addition, all reviews reported presentation of data in trials in a format that is currently not suitable for meta-analysis in RevMan 4.2 software, for example, the results of ANOVA, MANOVA and MANCOVA.

Heterogeneity and diversity

All reviews reported diversity of participants between, and sometimes within, trials, including differences in participant age and diagnosis. Two reviews reported baseline differences (AIT and Secretin) for intervention and placebo groups. The AIT review reported high levels of heterogeneity in meta-analyses. The authors of the secretin review reported concerns about non-significant baseline differences in trials with small sample sizes and used mean difference in meta-analysis where possible, but elected not to publish the results of any meta-analyses due to high heterogeneity. High heterogeneity was also reported in the risperidone review.

Use of a variety of outcome measures

For many systematic reviews included in this overview, there were multiple outcome measures used with very few outcome measures used by more than one trial for any one intervention. For example, in the AIT systematic review 17 outcome measures were used and only two of these were used by three or more studies, and one of these of questionability validity (see below). In the secretin review, 25 outcomes were reported in 14 trials.

The use of an outcome measure of questionable validity was reported in the AIT review. In two AIT trials using the Aberrant Behaviour Checklist (ABC)43 a ‘total score’ was calculated from a combination of five subscales scores. The tool is, however, designed to report the five subscales separately. The ‘total score’ has not been validated and any evidence of effect is also invalid. Neither the AIT review, nor this overview, includes those results.

Sample size and power

All reviews included small trials, with no trial including more than 150 participants per treatment group. Trial sample sizes of less than 20 were common. The total sample size for meta-analyses in reviews ranges from 20–208 (Tables 4 and 5).

Table IV. Outcomes
 Outcome measureOutcome subgroupIntervention and comparisonNo. participants (no. trials)Measure of treatment effect (95% CI)Quality of evidenceComments
  1. Negative direction of effect indicates improvement for the Parenting Stress Index and Aberrant Behavior Checklist. For all other outcomes positive direction of effect indicates improvement.

BehaviourAberrant Behavior ChecklistIrritabilityRisperidone vs placebo178 (2)− 8.09 (−12.99,− 3.19)LowHigh heterogeneity I2 = 77.75
  Social withdrawal/lethargyRisperidone vs placebo178 (2)− 3.00 (−5.03, − 0.97)Low 
  HyperactivityRisperidone vs placebo178 (2)− 8.98 (−12.01, − 5.94)Low 
  StereotypyRisperidone vs placebo178 (2)− 1.71 (−2.97, − 0.45)Low 
  Inappropriate speechRisperidone vs placebo178 (2)− 1.93 (−3.79,− 0.07)Low 
Overall impressionClinical Global Impression Scale Risperidone vs placebo208 (3)RR 4.83 (2.21, 10.59)LowDichotomized by trialists, ‘not improved vs improved/very much improved’, moderate heterogeneity (I2 = 43%)
CommunicationCommunication skills: gestural Music therapy vs placebo20 (2)SMD 0.50 (0.22, 0.79)Low 
 Communication skills: verbal Music therapy vs placebo20 (2)SMD 0.36 (0.15, 0.57)Low 
 MCDI words said Parent-mediated early intervention vs placebo52 (2)69.66 (2.39, 136.94)Low 
 MCDI words understood Parent-mediated early intervention vs placebo52 (2)75.84 (10.96, 140.72)Low 
Parental StressParenting stress index Parent-mediated early intervention vs placebo48 (2)− 3.45 (−15.94, 9.05)Low 
Table V. Adverse events
OutcomeIntervention and comparisonNo. participants (no. trials)Measure of treatment effect (95% CI)Quality of evidenceComments
Weight gainRisperidone vs placebo179 (2)WMD 1.78 (1.15–2.41)Low 

Length of follow-up

Few trials collected outcomes beyond three months. The review of music therapy38 reported outcomes after only one week. The longest period of data collected was for the AIT and gluten- and casein-free diet reviews, which both reported some outcomes at 12 months.

Adverse events

All reviews described poor reporting of adverse effects in trials with only one review (secretin) reporting one trial that used a well-known and standardized measure, the Treatment Emergent Symptoms Scale. No review could draw adequate conclusions about the risk of adverse effects for any of the interventions.

Effects of interventions

Many outcome measures sought by reviews were not reported in included trials. Few meta-analyses were conducted due to the use of varied outcome measures, concerns about methodological quality and/or presentation of data in a format not suitable for analysis in RevMan 4.2. Tables 4 and 5 present the results of meta-analyses of outcomes (Table 4) and adverse events (Table 5) reported in included studies.

No data syntheses were presented in the auditory integration training41, secretin40, gluten- and casein-diet32 or Vitamin B6 and magnesium38 reviews. Meta-analyses of diagnostic tools that assessed diagnostic features of autism were not available in any review.

Data synthesis for the Aberrant Behaviour Checklist (five subscales) were available from the Risperidone10 reviews. The irritability, hyperactivity, stereotypy and inappropriate speech subgroup scores were all statistically improved in the group who received risperidone therapy compared to placebo. As shown in Table 3, the effect size for these statistically significant differences vary for different ABC subgroup scores. The clinical relevance of the improvements in score for the treatment groups, as reported here, needs to be thought about in the context of both the magnitude of the between-group score difference and the baseline level of dysfunction of the children included in the trials, before the applicability of these findings can be evaluated. The quality of evidence is low, with evidence of moderate heterogeneity.

The risperidone review also reported meta-analysis of the Clinical Global Impression Scale, which gives an estimate of the severity of illness and clinical improvement. Low scores indicate decreased severity10. Meta-analysis showed a significant improvement in the Risperidone group with a four-fold between group difference. Again quality of evidence was low and heterogeneity moderate for this meta-analysis.

Gestural and verbal communication skills were measured in two trials included in the music therapy review using in-house designed scores counting numbers of gestures42. SMD was calculated as the scores differed between the trials, but measured the same construct. While the review reports a significant difference from meta-analysis, the trial sample sizes are small and the quality of the evidence is low, particularly due to the subjective nature of the assessment and the reported ambiguity of the criteria.

Communication outcomes, using mean CDI words said and understood, were also available for the PMEI review31, with improvements in the intervention groups compared to the placebo groups for the two trials that reported this outcome measure. The intervention group were saying and understanding approximately 70 words more over 12 months, but with considerable variation within the groups indicated by the wide standard deviations. This review also reported meta-analysis of between group differences for the Parenting Stress Index from two trials, and the difference was not statistically significant.

A statistically significance increase in weight gain was reported in the Risperidone review (WMD 1.78, 95% CI 1.15–2.41).

No review found sufficient high quality data to conclude that any intervention was an effective treatment for autism spectrum disorders. Some reviews (Risperidone, Music therapy and Parent-mediated behaviour interventions) provided evidence of statistically significant improvement effects for communication, speech and/or some behaviours associated with autism.

Discussion

Summary of main results

Overall completeness and applicability of evidence

This review provides a synthesis of current evidence from The Cochrane Library for treatments of autism spectrum disorders. As well as providing information for use in a clinical setting, it highlights concerns with the state of research into treatments for autism.

All of the authors of the reviews included in this overview conclude that, for the autism treatment they reviewed, there is a lack of high-quality, adequately powered randomized controlled trial evidence, for outcomes measured over a useful time period. All included reviews call for more and better evidence.

Only three of the included reviews, Risperidone, Parent Mediated Early Intervention and Music therapy, presented meta-analyses of results and show at least one statistically significant effect of treatment. The value of these significant treatment effects is discussed later.

For most reviews and for most outcome measures there was a lack of statistically significant results. Several things, including low power due to small sample size, may have caused this. However, concerns about outcome measures most greatly affect interpretation of the results of trials of interventions for ASD. For example, a lack of consistency in the use of outcome measures reduces the ability of authors of autism reviews to synthesize data and form conclusions about treatment efficacy. Data synthesis could be particularly useful in an area such as autism, where sample sizes for individual trials are frequently low. Of further concern is the use and availability of outcome measures that are suitable for monitoring change over time, especially change over short time intervals usually reported in trials49. Trials predominately relied on the use of diagnostic scales that are not designed or validated to measure change, such as the Childhood Autism Rating Scale. The Aberrant Behavior Checklist has shown some ability to measure change over time43 and this could be further explored, but the tool must be used correctly and consistently to obtain usable data.

Outcomes that are considered of importance, such as quality of life, were not reported in the trials that were included in the systematic reviews of this overview. While quality of life, economic analysis and parent satisfaction may be difficult to examine, they provide a big-picture evaluation that may be extremely useful to families and clinicians when making decisions about which treatments to use. Researchers need to commit to qualitative research to determine which outcomes are useful and relevant to consumers, clinicians and service providers.

Three interventions show statistically significant improvement, but there are limitations in the evidence for all three, as discussed below.

The review of music therapy reported a small improvement across two outcomes, verbal and gestural communicative skills. The relevance of these outcome measures is questionable. Outcomes do not relate to behavioural skills and there is no indication that the outcomes measuring communication are reflective of any ability to communicate outside the assessment sessions50. Sample sizes are very small and the majority of the weight of the evidence came from a single study of small sample size (n = 10). Despite the small sample size this study has a narrow confidence interval, suggesting little spread of results. The generalizability of the results is doubtful. Further, the study is a cross-over trial with great potential for contamination by carry-over of treatment effects.

Parent-mediated early intervention is a widely practiced intervention. The only statistically significant results available from meta-analysis for this intervention were the MCDI words said and understood. There was no improvement in parent stress as measured by the Parent Stress Index. Results from included trials show some improvement in adaptive behaviour but are limited by sample size. No subgroup analyses were possible and in this, as well as other interventions for autism, it is possible that this intervention is more useful when administered to very young children. As such, further trials should ensure sufficient sample power to allow subgroup analysis of data by age.

Results of meta-analysis for risperidone therapy showed statistically significant improvements in the Clinical Global Impression Scale and four of five domains of the Aberrant Behavior Checklist, despite small sample size. However, there was also statistically significant evidence of weight gain. It should be noted that the age range for the participants in the included trials ranged from 5–43 years. No subgroup analysis was available based on age and therefore effectiveness and adverse events profiles are not available for children. For any future use of risperidone, a risk benefit analysis is required before this treatment can be widely recommended. In children in particular, a tailoring of the therapy based on potential risks and benefits and with individual monitoring for positive and adverse therapy effects will always be necessary.

The interventions covered in this review are diverse. However, there are important and widely used interventions that are not included in this overview because Cochrane reviews are neither published nor in preparation for these topics. Interventions not already evaluated in Cochrane reviews include ecological or environmental modifications, skills-based developmental programs, visual supports, social skills development programs, behaviour modification programs, educational programs and comprehensive programs. It is possible that Cochrane systematic reviews do not exist for these interventions because of their anticipated complexity and hence the time commitment required for their completion. Systematic reviews of some of these therapies have been published outside The Cochrane Library, but are outside the scope of this overview. Also awaited is the publication of a Cochrane review on selective serotonin re-uptake inhibitors which is currently in protocol form.

The reader needs to interpret the findings of this overview in light of the inherent complexity of the interventions, particularly non-pharmacological approaches of treatment, that were evaluated in reviews and primary studies. Some of the interventions such as music therapy and parent-mediated early intervention have varied components that may be implemented in different ways and by different people including both professionals and lay people. Further, even pharmacological interventions can include different drugs within the medication class and may include different doses and frequency of administration (e.g. secretin). In some cases, this prohibits generalizations regarding a specific intervention and applicability in different settings.

The length of follow-up must be appropriate to the nature of the intervention and its mechanism of action (i.e. how long an intervention is required to begin to have an effect) and the outcomes being measured (i.e. length of time to elicit change in a specific outcome). Further, consideration needs to be made for whether or not any observed changes are maintained over the longer term.

Adverse events were poorly measured and reported in trials and hence in the reviews included in this overview. However, expected weight gain with risperidone therapy was confirmed.

As yet there has been little attention to the costs of these therapies in trials and hence reviews. Methods for inclusion of health economic data in Cochrane and Campbell systematic reviews are currently in development.

Quality of the evidence

All reviews included in this overview cited serious concerns with the quality of trials of interventions for autism. Sample sizes were typically very small, which increases the likelihood of type II error, that is, not finding significant results when they do exist. This problem is exacerbated by the use of many different outcome measures across trials which means that meta-analysis cannot be used to synthesize this evidence. As a result, even following systematic review, power remains low. Thus, it is not possible to be sure whether treatments that have been subjected to trials are effective or not.

The lack of ability to detect real change following treatment is compounded by the use of outcome measures that may not be suitable for detecting change, especially over short time periods. In addition, the lack of sample size adequate has prevented the estimation of important subgroup effects in reviews. As described above, there are situations in which further analysis for age is necessary to provide information that can inform clinical decision-making about treatment application. Further, there have been suggestions that some subgroups of children with autism may respond to certain therapies and others not. For example, it has been suggested that only children with gastrointestinal disturbance or an onset of autism marked by loss of skills will respond to secretin therapy. However, researchers have not been able to explore this theory using subgroup analysis.

Although data in trials of ASD are often analysed appropriately, the results are not presented in form suitable for meta-analysis using Cochrane software. For example, trials with baseline differences between treatment and placebo groups have rightly chosen analysis of variance51 but that data is not able to be included in meta-analysis. This problem is currently being addressed.

As yet, only short-term outcome data are available for the treatments included in this review that show most promise in improving some of the problems associated with autism.

Potential biases in the overview process

Danielle Wheeler and Katrina Williams are authors of two of the reviews included in this overview. Both are editors for The Cochrane Developmental, Psychosocial Learning Problems Group and have edited reviews and/or updates of several other included reviews.

Agreements and disagreements with other studies or reviews

None.

Authors' Conclusions

Implications for practice

There is as yet no clear evidence of benefit following treatment for ASD reported in any Cochrane review. Research in autism is in a period of growth and development, and more good quality trials are needed to clarify if this means that the treatments that are the subject of this overview are not effective or that there is insufficient evidence to show true treatment effectiveness.

Statistically significant improvements with risperidone treatment for the Clinical Global Impression Scale and for some subgroups of the ABC need to be considered in terms of the magnitude of the clinical improvement and balanced against potential side effects such as weight gain.

Statistically significant improvements in communication and speech for the parent-mediated behavioural interventions and music therapy warrant further examination.

Implications for research

There is now an urgent need to conduct high quality randomized controlled trials and systematic reviews of the most commonly used interventions for ASD. Researchers must attend closely to trial quality and ensure the use of valid outcome measures that are sensitive to change and measure outcomes that are important to individuals and their families. There needs to be a dialogue between international researchers which ensures that outcome measures are used consistently across trials, allowing data synthesis by reviews. This will improve sample power for future systematic reviews and assessment of treatment efficacy.

Acknowledgements

The authors wish to thank Jane Dennis of The Cochrane Developmental, Psychosocial and Learning Problems Group, for doublechecking the data extraction, as well as Lorne Becker and The Cochrane Umbrella Reviews Working Group for their assistance with methodology. Thank you to the Cochrane review authors who supplied us with updated reviews in time for inclusion in this overview.

Contributions of Authors

Danielle Wheeler conducted the search, extracted the data and wrote the text of the review. Katrina Williams provided advice about the approach to the methodology of this review and wrote the text of this review. Jennifer Seida provided a second independent assessment of the quality of the systematic reviews, and contributed to the text of this review. Maria Ospina contributed to the text of this review.

Conflict of Interest

None known.

Ancillary