Criteria for considering studies for this review
Types of studies
We included randomised controlled trials that evaluated the effects of BIs on substance use as well as on other behavioural outcomes associated with adolescent substance use. Studies that recruited adolescents from anywhere else other than an educational setting were excluded.
Types of participants
Participants were adolescents under the age of 19 who were attending high school, secondary school, or a further education training college that provided alternative schooling or vocational training for adolescents between the ages of 16 and 18 years, and who used or misused alcohol or other drugs, or both, but did not meet the criteria for substance dependency. In addition, participants had faced negative behavioural consequences due to their substance use.
Types of interventions
The intervention should have been labelled as a BI, but could also have been defined as motivational interviewing, brief skills-orientation, motivational enhancement, or other specific types of BIs, which were up to four sessions long and used BI principles to facilitate change. The focus should have been on building the individual's motivation to change. The BIs could have been offered as a stand-alone option, integrated with other intervention efforts, or as a precursor to other treatments. Only BIs that were offered to individuals in a face-to-face modality were included in this review.
The control could have been no intervention, placebo, assessment only, or other types of interventions or education.
Types of outcome measures
Abstinence or reduction of substance use behaviour.
The outcome measures could have been self-reported measures, including dichotomous and continuous outcomes. In addition, substance use could have been measured with standardised measures of substance use that are appropriate for adolescents such as the Alcohol Diagostic Interview (ADI), Adolescent Drug Abuse Diagnosis (ADAD), Adolescent Drug Involvement Scale (ADIS), Adolescent Alcohol and Drug Involvement Scale (AADIS), and Personal Experience Inventory (PEI), which are generally self-report measures.
Any biological testing could also have been included, such as urinalysis for drug use and breathalyser tests for alcohol use.
Engagement in criminal activity (such as theft, drug and alcohol crimes, property crimes) related to substance use.
Engagement in delinquent-type behaviours (such as drinking and driving, aggression and fighting, bullying, carrying weapons to school, buying and selling drugs, gang involvement, truancy, suspension and expulsion, and disobeying rules in general) related to substance use.
The secondary outcomes refer to problem behaviours related specifically to delinquent-type behaviours.
It was not expected that the included BIs would have adverse effects on the primary or secondary outcomes.
Search methods for identification of studies
Included studies were published from 1966 onwards, the year that BIs were first introduced.
Relevant trials were obtained from searching the following sources:
1) Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library (2010, Issue 3) which includes the Cochrane Drug and Alcohol Group Specialised Register;
2) PubMed (January 1966 to March 2013);
3) PsycINFO (January 1966 to March 2013);
4) ERIC (Education Resources Information Centre) (January 1966 to March 2013);
5) ISAP (the Index of South African Periodicals), Social Science Index (January 1966 to March 2013);
6) Academic Search Premier (January 1966 to March 2013);
7) EMBASE (1974 to March 2013);
8) LILACS (2004 to March 2013)
9) Alcohol and Alcohol Problems Science Database (1972 to March 2013);
10) Social Science Citation Index (January 1966 to March 2013).
We developed a detailed search strategy for each database. The search strategy combined the subject search with the Cochrane Highly Sensitive Search Strategy (CHSSS) for identifying randomised trials in PubMed, sensitivity maximising version (2008 revision), as referenced in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).
The subject search utilised a combination of controlled vocabulary and free text terms based on the search strategy for searching PubMed. This search strategy was adapted as appropriate for the other databases (see Appendix 1; Appendix 2; Appendix 3; Appendix 4; Appendix 5; Appendix 6; Appendix 7 for all searches). No language restrictions were applied.
Searching other resources
We contacted relevant authors and searched citations in all relevant papers to obtain information on potential additional randomised controlled trials. In addition, the authors searched for other unpublished studies and assessed relevant conference proceedings for additional references. The following websites were also searched:
Current Controlled Trials (http://www.controlled-trials.com/);
Data collection and analysis
Selection of studies
All the papers from the electronic searches were assessed by TC and BM to identify potentially eligible studies, and the full texts retrieved. Selection from this initial search was based on information derived from the title, abstract and keywords.
These included randomised controlled trial or clinical trial and substance use, alcohol use, drug use (and related terms), alcohol or drug use or substance use reduction strategies (and related terms), problem behaviours (including but not limited to aggression, fighting, suspension, expulsion, weapon-carrying), interventions, school staff or settings or both (and related terms).
If the title, abstract and keywords did not provide enough information to make an informed decision with regards to inclusion of the paper, the full text of the paper was obtained.
The full texts of potentially relevant studies were assessed for inclusion by TC and BM. While there was no disagreement on the inclusion of studies, a third review author was on hand (JL) to resolve any disagreements.
Data extraction and management
Data were independently extracted by two independent authors (TC and BM) using a piloted data extraction form based on the Cochrane Collaborative Drugs and Alcohol Review Group's extraction form and subsequently entered in the Cochrane Collaboration software (Review Manager 5.1) for analysis (data extraction form available on request from Carney). Data on the following information were extracted from studies: study design and method, allocation process, participant data, intervention, and outcomes. When there was missing information from the original studies on outcomes or other important information, we contacted the corresponding author via e-mail in order to request additional data. Certain statistics were not readily available in the articles, and if authors were not able to provide this information to the authors we calculated them from existing data using the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) for guidance.
Assessment of risk of bias in included studies
Two review authors independently assessed potential biases resulting from the trial design. Any discrepancies between the review authors were resolved by discussion. Quality assessment was based on the following aspects of methodology: random sequence generation, allocation concealment, blinding of outcome assessor, completion of outcome data, selective outcome reporting, and additional biases (Higgins 2011). Firstly, a description of the various areas of possible biases was entered into a data extraction form. Secondly, these were assigned a judgement of 'low', 'high' or 'unclear' risk of bias. If the two review authors struggled to make a judgement, we contacted the author of the article in an attempt to gain more information about the particular bias domain, and only if it was still unclear was it assigned a judgement of 'unclear'.
Blinding of personnel and participants was impossible due to the type of intervention delivered, and therefore this item relating to risk of performance bias was deactivated in the risk of bias table (see Appendix 8). Only the blinding of the outcome assessor was assessed in this review. For other risks of bias, the following were examined:
For a detailed description of the criteria used to assess risk of bias, please see Appendix 8.
Measures of treatment effect
We compared the outcomes of the experimental and control groups at different follow-up appointments. Originally we planned to categorise the findings into short-term follow-up appointments (one to three months), medium-term follow-up appointments (four to 11 months) and long-term follow-up appointments (12 months and longer). However, this was not possible because there were very few studies that could be included. For studies that reported on more than one follow-up period, we used the last follow-up period. Dichotomous outcome measures were assessed by calculating the risk ratio (RR) with the 95% confidence interval (CI), while for continuous outcome measures the standardised mean difference (SMD) with 95% CI was the treatment measure used as the summary statistic. Commonly in meta-analysis the studies assess the same outcome but measure it in a variety of ways, so the same outcome may be measured with different scales (Higgins 2011). If standard deviations for the mean values were not provided, we used the standard errors that were provided and used the calculation in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) to change them to standard deviations.
The quality of evidence was rated using the GRADE approach (Balshem 2011). The GRADE approach defines the quality of evidence for outcomes as the extent to which one can be confident that the estimate of an effect is close to the quantity of interest. The following criteria were used to rate the quality of evidence for selected outcomes: 1) methodological quality, which considers the risk of bias within studies; 2) directness of evidence, by addressing population, intervention and outcome of interest; 3) inconsistency (significant heterogeneity which cannot be explained); 4) imprecision (if the 95% CI includes appreciable benefit or harm); and 5) publication bias, which involves study sample size and whether or not studies were commercially funded.
Unit of analysis issues
The analysis of clinical trials needs to take into account the level at which randomisation occurred. While this can be on an individual basis, cluster-randomised trials have groups of individuals (for example schools, community) as opposed to individuals as the unit of analysis. The review authors originally planned to measure the intra-cluster correlation coefficient (ICC) in these studies and then use the ICC to measure the design effect, which is an inflation factor that is used to increase the statistical power of the study (Campbell 2000). However, the authors of the cluster-randomised trials used the Huber-White estimator of variance to control for the effects of clustered recruitment, and further calculations were therefore not necessary. While the review authors had decided to use a conversion rate of 4.29 (30 days/7) where outcomes across studies used different measurement times other than monthly frequency, it was unnecessary to do any additional conversions as the measures in the studies were of monthly use (for example frequency of use, quantity of use).
Dealing with missing data
We contacted the original investigators of the included studies up to three times to request any missing data (missing studies, outcomes, summary data, individuals, and study-level characteristics). We needed to decide whether the data were missing at random (not related to the actual data) or not missing at random (related to the actual data). When study data were assumed to be missing at random, only the available data were analysed. For data that were not missing at random, this needed to be addressed by performing a sensitivity analysis or, if this was not possible, by replacing missing data with specified values (Higgins 2011). The imputation of missing data with specific replacement values was not needed for the studies included in this review.
Assessment of heterogeneity
We assessed the extent of heterogeneity across the studies using the Chi² (X²) test and I² statistic and looking at whether the P values were statistically significant (Higgins 2011), with a P value of 0.10 or less showing significant heterogeneity.
Assessment of reporting biases
We planned to use funnel plots (plots of the effect estimate from each study against the sample size or effect standard error) in an attempt to assess any publication bias. More specifically, we planned to examine the funnel plots for asymmetry as an indication of publication bias. However, asymmetrical funnel plots are not always caused by publication bias and publication bias does not always cause asymmetrical funnel plots (Higgins 2011). This was not possible for the current review because less than 10 studies were included.
A meta-analysis was performed as there were more than two individual trials with comparable intervention methods and outcomes that could be analysed. Random-effects models were used based on the fact that we expected different types of interventions to be included in the review and combined in the meta-analysis (such as interventions of different duration and using different follow-up measures).
Subgroup analysis and investigation of heterogeneity
Although the authors originally planned to conduct subgroup analyses for studies with low and unclear risk of bias and, if possible, for different ages, gender and school grades for adolescent study participants, this was not possible. There was only a small number of studies included in the meta-analysis, and the results were not reported by these variables of interest.
The authors decided that if there was significant unexplained heterogeneity and more than 10 studies were included in the analysis, they would perform a sensitivity analysis to consider if the following had an impact on effect size:
1. studies conducted in settings other than traditional high or secondary schools (e.g. alternative high schools, reform school);
2. studies which utilised quasi-experimental designs (as long as an experimental and a control group were included);
3. studies which had attrition rates of more than 20%.
Since only six studies were included in the review, these sensitivity analyses were unnecessary.