Editorial: Prevention in the community and treatment in the clinic – two different methodological approaches to determining the evidence-base for child mental health interventions


  • Tony Charman

This Editorial focuses on two papers in the current issue of the Journal. Both are concerned with moving forward the evidence-base for child and adolescent mental health interventions and both have a number of features that make their contribution particularly strong.

Webster-Stratton and colleagues report the findings from a large community prevention programme that aims to improve “school readiness”– a construct that is know to be strongly positively predictive of longer-term academic progress and negatively predictive of later behaviour problems. Watson and Rees conduct a meta-analysis of clinic-based randomised controlled treatment trials for paediatric obsessive-compulsive disorder (OCD), examining effects both for pharmacological treatments and for cognitive behavioural treatments (CBT). Both papers, by necessity, involve the use of complex statistical approaches and contain detailed tabular and graphical presentation of the findings. Fortunately for those readers, including the present one, who sometimes struggle to follow every nuance of such complex analyses both papers provide clear and digestible summaries of their main findings. They also set out clear policy and practice agendas, respectively, that should influence social policy and clinical practice in many countries internationally.

Webster-Stratton and colleagues’ study included a very large (= 1768) sample of kindergarten and first grade children in school areas selected for high rates of poverty: the sample was ethnically very diverse; almost one-third of children did not speak English as their mother tongue; and over half were in receipt of free school lunches. Hence, this was a community sample at high risk of poor outcomes. Designed as a community prevention programme, the study applied a teacher-delivered child training programme that previously has been demonstrated to be effective in controlled trials of clinic-identified children with behaviour problems. In the schools randomly allocated to the treatment arm of the study all children in the classrooms received the classroom-based intervention (which was ethically permissible due to its positive evidence-base and negligible risk of harm), although only children whose parents had consented were assessed (over three-quarters of parents opted in). In part due to its size, the study was conducted across four consecutive school years but this also allowed schools that had served as controls in one year to receive the intervention programme the following year (in part akin to a “wait list” control in clinic-based trials). The complex modelling approach to the statistical analysis took account of the nesting or clustering of children within classrooms, with a particular teacher or teachers, within a particular school. One particular strength of the study was that in contrast to many studies that aim to decrease problematic behaviour and improve children’s competence that have relied on parent and teacher ratings of children’s behaviour, Webster-Stratton et al. used (blinded) classroom observations of both teachers and children. Whilst this is a resource-intensive decision in terms of data collection and scoring, it provides a robust and valid measure of behaviour in the classroom.

The study found positive effects in terms of more positive teacher management strategies and improved social competence, emotional self-regulation and fewer conduct problems in the children in the schools receiving intervention compared to controls. The effects were moderate to large and there was evidence that the changes were largest for the children with the poorest initial scores – perhaps those one would most want to target in such an at-risk community sample. Such large and ambitious community preventative programmes are time-consuming and costly but are crucial to informing progressive social policy decisions. The findings need to be brought to the attention of policy makers in the health and education fields internationally. This study does an impressive job of taking a treatment approach with a good evidence-base out of the clinic and into deprived communities. However, it still remains one step away from an absolute demonstration of community effectiveness since the classroom-based curriculum was co-led by research staff. Whether this would be true for a programme staffed only by educational practitioners remains an outstanding question.

Watson and Rees’ meta-analysis is the one of the few conducted to date on paediatric OCD to include only randomised trials, where previous meta-analyses have pooled randomised and uncontrolled studies. The importance of this approach is that non-randomised trials are likely to inflate estimates of the true treatment effect due to increased risk from “threats to validity”, such as regression to the mean and patient expectations. The authors also adopted a sophisticated approach to their analysis by examining additional meta-analytic statistical factors. The first of these was publication bias (trials with null effects might be less likely to be published – the rationale behind the recent establishment of clinical trial registries). The second and third were heterogeneity and sensitivity analyses that, respectively, examine the extent to which treatment outcome varies across studies and the extent to which results differ when inclusion criteria are varied. These latter statistics determine whether the findings from a meta-analysis are likely to be generalisable and valid.

From this analysis some clear findings emerged that should form the basis for recommended clinical practice. Both psychopharmacological treatments and CBT were shown to be effective. Whilst the effect size was larger for the trials that examined CBT treatments than for those that examined pharmacological treatments, both analyses were statistically significant, with the effect for pharmacological studies being most significant. This is explained both by the fact that there were more effect size estimates for the latter (10 including 1016 patients) than the former (5 including 161 patients) and the fact that confidence estimates were broader for the CBT treatment comparisons, reflecting the greater variation in effect size estimates across the studies conducted. Although the effect size estimates were large for CBT and moderate for pharmacological treatments, they were lower than those reported in previous meta-analyses that have included uncontrolled trials – justifying the conservative approach the authors adopted by only including randomised studies.

Watson and Rees conclude that CBT and pharmacological treatments are the recommended first-line treatments for paediatric OCD. However, the relatively restricted pool of randomised trials on which the authors were able to draw – and the added limitations of the predominance of single-site trials in the CBT literature and the range of individual drugs tested across different pharmacological studies – means that further work is required. In particular, moderator effects – determining the patient characteristics that predict response to a particular treatment – could not be examined on the basis of the number of randomised trials reported to date in the literature. Whilst this study confirms that there is a good evidence-base to direct treatment of paediatric OCD, further randomised trials of both treatment approaches are required over the coming years.