Intervention Review

You have free access to this content

Media campaigns for the prevention of illicit drug use in young people

  1. Marica Ferri1,*,
  2. Elias Allara2,
  3. Alessandra Bo1,
  4. Antonio Gasparrini3,
  5. Fabrizio Faggiano4

Editorial Group: Cochrane Drugs and Alcohol Group

Published Online: 5 JUN 2013

Assessed as up-to-date: 21 MAY 2013

DOI: 10.1002/14651858.CD009287.pub2


How to Cite

Ferri M, Allara E, Bo A, Gasparrini A, Faggiano F. Media campaigns for the prevention of illicit drug use in young people. Cochrane Database of Systematic Reviews 2013, Issue 6. Art. No.: CD009287. DOI: 10.1002/14651858.CD009287.pub2.

Author Information

  1. 1

    European Monitoring Centre for Drugs and Drug Addiction, Interventions, Best Practice and Scientific Partners, Lisbon, Portugal

  2. 2

    University of Torino, School of Public Health, Torino, Italy

  3. 3

    London School of Hygiene & Tropical Medicine, London, UK

  4. 4

    Avogadro University, Department of Translational Medicine, Novara, Italy

*Marica Ferri, Interventions, Best Practice and Scientific Partners, European Monitoring Centre for Drugs and Drug Addiction, Cais do Sodre' 1249-289 Lisbon, Lisbon, Portugal. marica.ferri@emcdda.europa.eu.

Publication History

  1. Publication Status: New
  2. Published Online: 5 JUN 2013

SEARCH

 
Characteristics of included studies [ordered by study ID]
Carpenter 2011

MethodsStudy design: interrupted time series study

Sampling: systematic sampling (schools are selected within geographic areas that are determined by the sampling section of the University of Michigan Survey Research Center, page 949)

Comparison group(s): pre-intervention surveys

Follow-up duration: n/a

Study time span: 2006 to 2008 (approximately 36 months)


Participants130,245 youths from 8th to 12th grade (13- to 18-year-old)


InterventionsAll media for 210 media markets for 2006 to 2008, after the introduction of the Above the Influence campaign


Outcomes
  • Past 30-day marijuana use
  • Lifetime marijuana use
  • Past-month alcohol consumption


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Unclear riskNot reported

Incomplete outcome data (attrition bias)
All outcomes
Unclear riskNot reported

Selective reporting (reporting bias)Unclear riskNo information regarding potential reporting bias

Colorado Meth 2011

MethodsStudy design: interrupted time series study

Sampling: n/a

Comparison group(s): pre-intervention survey

Follow-up duration: n/a

Study time span: March 2009 to April 2011 (26 months)


Participants1803 youths (600 + 601 + 602)


InterventionsMeth Project (USA), a "messaging campaign, supported by community outreach, and public policy initiatives". The campaign comprises "television, radio, print, billboard, and Internet advertising"


Outcomes
  • Past-month use of methamphetamine
  • Attitudes on methamphetamine and other drugs
  • Perceptions concerning methamphetamine and other drugs
  • Information sources and advertising awareness
  • Statewide Meth Project awareness and perceptions


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaires

Incomplete outcome data (attrition bias)
All outcomes
Unclear riskNot reported

Selective reporting (reporting bias)Unclear riskNo information regarding potential reporting bias

Czyzewska 2007

MethodsStudy design: randomised controlled trial

Sampling: not specified

Comparison group(s): 4 = 2 anti-tobacco advertisements x 2 orders of advertisements (i.e. explicit attitudes towards tobacco or marijuana)

Follow-up duration: not applicable

Study time span: not specified


Participants229 college students aged 18 to 19 years


Interventions15 advertisement embedded in a 15-minute science programme (USA). 10 advertisements were youth directed, 5 were non-youth directed. Each programme comprised of 90-second science film segments, 30-second youth-directed ad, 30-second non-youth-directed ad, then again another 30-second youth-directed ad. There were 4 versions of recorded programme corresponding to 4 experimental conditions: 2 types of advertisements (i.e. anti-tobacco or anti-marijuana) x 2 orders of advertisements (i.e. explicit attitudes towards tobacco or marijuana)


Outcomes
  • Implicit and explicit attitude towards tobacco
  • Implicit and explicit attitude towards marijuana


NotesImplicit attitudes were assessed through the Implicit Association Test (IAT)


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskp. 117 "They were randomly assigned to experimental conditions", but randomisation details are not reported

Allocation concealment (selection bias)Low riskAlthough full allocation concealment is not possible for this kind of study, there is low risk of selection bias because researchers administering the intervention were unlikely to know the children. See p. 117: "Two groups of 18- to 19-year-old college students were exposed to either anti-tobacco or anti-marijuana advertisements followed by implicit and explicit tests of attitudes to both, marijuana and tobacco"

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaire

p. 119 "Next to a computer, each person had a survey with a pre-recorded ID number on it"

Incomplete outcome data (attrition bias)
All outcomes
Low riskAll participants completed the test (being a post-only design); p. 117 "Two groups of 18- to 19-year-old college students were exposed to either anti-tobacco or anti-marijuana advertisements followed by implicit and explicit tests of attitudes to both, marijuana and tobacco"

Selective reporting (reporting bias)Unclear riskNo study protocol was mentioned

Fang 2010

MethodsStudy design: randomised controlled trial

Sampling: random sampling (through online advertisement and community service agencies)

Comparison group(s): no intervention

Follow-up duration: 6.25 months

Study time span: September 2007 to ˜December 2008 (˜16 months)


Participants108 Asian-American girls aged 10 to 14 with private access to a computer, and their mothers


InterventionsInternet-based prevention programme (USA) guided by family interaction theory and aiming to prevent girls' substance use through enhancing mother-daughter interactions. 9 sessions: mother-daughter relationship, conflict management, substance use opportunities, body image, mood management, stress management, problem solving, social influences, self efficacy. The programme was not designed expressly for Asian-Americans


Outcomes
  • Past 30-day use of
    • alcohol
    • cigarettes
    • marijuana
    • prescription drugs
  • Intention to use any of the above in the future
  • Depression
  • Other variables
    • Self efficacy
    • Refusal skills
    • Mother-daughter closeness
    • Mother-daughter communication
    • Maternal monitoring
    • Family rules against substance use


NotesOnly 1 post-test survey. Unclear whether the intervention focused on a single substance or many


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskp. 530 "Mother–daughter dyads were randomly assigned to intervention (n = 56) and control arms (n = 52)", but randomisation details are not reported

Allocation concealment (selection bias)Low riskAlthough full allocation concealment is not possible for this kind of study, there is low risk of selection bias because researchers administering the intervention were unlikely to know the children. See p. 530: "Delivered by voice-over narration, animated graphics, and games, session content involved skill demonstrations and interactive exercises that required the joint participation of mothers and daughters."

Blinding of outcome assessment (detection bias)
All outcomes
Low riskOnline questionnaire

p. 530 "Girls and mothers had separate and unique log-in names and passwords, and each completed a pretest and posttest survey online"

Incomplete outcome data (attrition bias)
All outcomes
Low riskLow number of missing outcome data; missing data balanced in numbers across study groups
p. 530 "Mother–daughter dyads were randomly assigned to intervention (n = 56) and control arms (n = 52)"
"Two mother–daughter dyads attrited from each arm, and 104 dyads (54 intervention and 50 control) successfully completed both pretest and posttest measures"

Selective reporting (reporting bias)Unclear riskp. 530 "The study protocol was approved by Columbia University's Institutional Review Board"

Fishbein 2002

MethodsStudy design: randomised controlled trial

Sampling: systematic random sampling (letters from each included middle/high school)

Comparison group(s): 5 experimental (6 advertisements each, embedded in a 24-minute documentary) + versus no intervention (documentary only) condition

Follow-up duration: not applicable

Study time span: not specified


Participants3608 youths aged 11 to 18 years (grades 4 to 12), median age 15 years


Interventions30 public service announcements produced by the Partnership for a Drug Free America (USA)


Outcomes
  • 30 dependent variables (5 scores for each of the 6 PSAs)
    • Perceived PSA effectiveness and realism
    • Negative and positive emotional response
    • Amount learned (understanding of intended message and on)
  • 5 scores resulting from mean of the 6 PSA scores
    • Total perceived PSA effectiveness and realism
    • Total negative and positive emotional response
    • Total amount learned (understanding of intended message and on)
  • Perceptions:
    • Perceived danger of engaging in risky behaviours
    • Perceived harmfulness of engaging in risky behaviours
    • Social norms


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNo mention of the sequence generation in the article (methods section p. 239)

Allocation concealment (selection bias)Unclear riskNo mention of the sequence generation in the article (methods section p. 239)

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaire

p. 240 "Confidentiality and anonymity were emphasized in the instructions, both in written and audio-video form"

Incomplete outcome data (attrition bias)
All outcomes
Low riskApparently almost all the sample exposed to interventions were included in the final analysis (and filled out the questionnaires). But no mention of the number originally enrolled, mention of some drop-outs apparently unlinked to outcomes but no absolute numbers reported

Selective reporting (reporting bias)Low riskProtocol not available but we do not suspect selective reporting

Georgia Meth 2011

MethodsStudy design: before and after study

Sampling: 4-stage probability sampling

Comparison group(s): pre-intervention survey

Follow-up duration: n/a

Study time span: November 2009 to April 2011 (18 months)


Participants4454 youths (2432 + 2022)


InterventionsMeth Project (USA), a "messaging campaign, supported by community outreach, and public policy initiatives". The campaign comprises "television, radio, print, billboard, and Internet advertising"


Outcomes
  • Past-month use of methamphetamine
  • Attitudes towards methamphetamine and other drugs
  • Perceptions concerning methamphetamine and other drugs
  • Information sources and advertising awareness
  • Statewide Meth Project awareness and perceptions


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaires

Incomplete outcome data (attrition bias)
All outcomes
Unclear riskNot reported

Selective reporting (reporting bias)Unclear riskNo information regarding potential reporting bias

Hawaii Meth 2011

MethodsStudy design: interrupted time series study

Sampling: 4-stage probability sampling

Comparison group(s): pre-intervention survey

Follow-up duration: n/a

Study time span: March 2009 to March 2011 (25 months)


Participants3305 youths (1065 + 1035 + 1205)


InterventionsMeth Project (USA), a "messaging campaign, supported by community outreach, and public policy initiatives". The campaign comprises "television, radio, print, billboard, and Internet advertising"


Outcomes
  • Past-month use of methamphetamine
  • Attitudes on methamphetamine and other drugs
  • Perceptions concerning methamphetamine and other drugs
  • Information sources and advertising awareness
  • Statewide Meth Project awareness and perceptions


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaires

Incomplete outcome data (attrition bias)
All outcomes
Unclear riskNot reported

Selective reporting (reporting bias)Unclear riskNo information regarding potential reporting bias

Hornik 2006

MethodsStudy design: prospective cohort study

Sampling: systematic sampling (4-stage, geographic)

Comparison group(s): lower exposure to intervention

Follow-up duration: November 1999 to June 2004 (56 months)

Study time span: September 1999 to June 2004 (58 months). Up to 4 observations per each of the 3 cohorts. Interviews were carried out at home


Participants8117 youths aged 12.5 to 18 years in the first round


InterventionsThe National Youth Anti-Drug Media Campaign (USA) was a comprehensive social marketing campaign aimed at youths aged 9 to 18 years and disseminated though television, radio, websites, magazines, movie theatres and others. The campaign established partnership with civic, professional and community groups and outreach programs with the media, entertainment and sport industries


Outcomes
  • Lifetime, past-year and past 30-day use of marijuana
  • Intention to use marijuana
  • Attitudes towards marijuana and self efficacy to resist use of marijuana
  • Perceptions and social norms about marijuana


NotesNIDA report 'Evaluation of the National Youth Anti-Drug Media Campaign: 2004 Report of Findings. June 2006', on which this article is based, was also used to retrieve information for this meta-analysis


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous and administered via computer

p. 2230 (Hornik 2008) "NSPY questionnaires were administered on laptop computers brought into the respondents' homes. The interviewer recorded answers for the opening sections, but for most of the interview, to protect privacy, respondents heard pre-recorded categories of questions and answer through headphones and responded via touch screen selection on the computer. Interviews could be conducted in English or Spanish"

Incomplete outcome data (attrition bias)
All outcomes
Low risk"The overall response rate among youths for the first round was 65%, with 86% to 93% of still eligible youths interviewed in subsequent rounds", page 2230 in Evaluation of the National Youth Anti-Drug Media Campaign: 2004 Report of Findings, page 2-12, table 2-A "Completed interviews by wave"

Selective reporting (reporting bias)Low riskNo protocol available but the we do not suspect selective reporting bias

Idaho Meth 2010

MethodsStudy design: interrupted time series study

Sampling: 4-stage probability sampling

Comparison group(s): pre-intervention survey

Follow-up duration: n/a

Study time span: September 2007 to December 2010 (40 months)


Participants11,143 youths (3091 + 2590 + 2641 + 2821)


InterventionsMeth Project (USA), a "messaging campaign, supported by community outreach, and public policy initiatives". The campaign comprises "television, radio, print, billboard, and Internet advertising"


Outcomes
  • Past-month use of methamphetamine
  • Attitudes towards methamphetamine and other drugs
  • Perceptions concerning methamphetamine and other drugs
  • Informations sources and advertising awareness
  • Statewide Meth Project awareness and perceptions


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaires

Incomplete outcome data (attrition bias)
All outcomes
Unclear riskNot reported

Selective reporting (reporting bias)Unclear riskNo information regarding potential reporting bias

Kelly 1992

MethodsStudy design: randomised controlled trial.

Sampling: not specified.

Comparison group(s):

  • control group (no anti-drug PSA and no group discussion).
  • experimental group 1 (anti-drug PSA without group discussion)
  • experimental group 2 (anti-drug PSA with group discussion)


Follow-up duration: 1.5 months (6 weeks).

Study time span: not specified, at least 1.5 months.


Participants79 female college students, primarily 18 to 19 years old


InterventionsAnti-drug messages (USA) selected from the library of the Media Advertising Partnership for a Drug-Free America and centred on drugs and alcohol


Outcomes
  • Attitudes towards marijuana
  • Attitudes towards cocaine
  • Attitudes towards crack
  • Attitude towards getting drunk


NotesPre-test, post-test and 6-week follow-up means are provided. Standard deviations are not provided


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot reported
p.80 "Subjects were randomly divided into a total of 9 discussion groups."

Allocation concealment (selection bias)Unclear riskNot reported
Baseline comparisons reported

Blinding of outcome assessment (detection bias)
All outcomes
Unclear riskBlinding of outcome assessors not reported

Incomplete outcome data (attrition bias)
All outcomes
Low riskNo missing data and clear reporting of sample size both of the intervention and control group

Selective reporting (reporting bias)Low riskNo study protocol available but clear reporting of main study hypothesis and direct correlation between main topics investigated in the experiment and reported outcomes
TOPICAL EXPERIMENTAL AREAS:
p. 79 "two topical areas chosen for the study were (1) the age at which parents should talk to their children about dangers of drugs, (2) how much responsibility one has, if any, for another's drug use"
OUTCOMES:
p. 80 "three questions asking at what age children should be spoken to about marijuana, cocaine and crack"
p. 81 "one question asked subjects to rate their agreement on a 5 point Likert scale with the statement "whether or not I get drunk is nobody's business". Similar question were asked regarding use of marijuana, cocaine and crack"

Lee 2010

MethodsStudy design: randomised controlled trial

Sampling: random sampling (letters and email sent to ˜4000 college students at a "large public university in the Northwest United States")

Comparison group(s): no intervention (no feedback or information, students were asked to complete web-based assessments)

Follow-up duration: 6 months

Study time span: June 2005 to not specified (at least 6 months because a 6-month follow-up was performed)


Participants341 college students aged 17 to 19 with any use of marijuana in the 3 months before study


InterventionsInternet-based personalised feedback intervention (USA). Participants were presented with feedback about their marijuana use, perceived and actual descriptive norms about marijuana use, and perceived pros and cons of using marijuana. Skills and training tips for avoiding marijuana and making changes in use were provided, as well as limited alcohol feedback. Perceived high-risk contexts and alternative activities around campus and in the communities were provided


Outcomes
  • Past 90-day use of marijuana
  • Contemplation to change marijuana use (intention)
  • Consequences of marijuana use (knowledge)
  • Family history of drug problem


NotesBaseline survey, then 3- and 6-month follow-ups


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Low riskComputer-based
p. 267, "Students were randomly assigned to a personalized feedback intervention (PFI) or control condition based on their screening responses (prior to baseline), using a stratified randomization procedure to produce groups with equivalent use rates at randomization"

Allocation concealment (selection bias)Low riskAlthough full allocation concealment is not possible for this kind of study, there is low risk of selection bias because researchers administering the intervention were unlikely to know the children. See p. 268: "Students in the intervention group received individual personalized feedback based on baseline information. On completion of the baseline survey, PFI participants could immediately view feedback online and could choose to print feedback to their own printer. Participants could return to view feedback on the web for 3 months"

Blinding of outcome assessment (detection bias)
All outcomes
Low riskComputer-administered questionnaire (p. 266-7)

Incomplete outcome data (attrition bias)
All outcomes
Low riskp. 268 "All analyses are based on intent-to-treat, regardless of whether participants viewed their feedback"

Selective reporting (reporting bias)Unclear riskp. 267 "All study procedures were approved by the university IRB and a federal Certificate of Confidentiality was obtained from the National Institutes of Health"

Miller 2000

MethodsStudy design: before and after study

Sampling: random sampling

Comparison group(s): no intervention (other campus with no intervention)

Follow-up duration: 1 year

Study time span: 1988-9, for 1.5 years


Participants1024 college students at baseline (median age 25 in the intervention group, 22 in the control group), 865 at 1-year follow-up


InterventionsThe Campuswide Alcohol and Drug Abuse Prevention Program (CADAPP; USA), based on self regulation theory. The campaign made use of printed materials, videotapes, speakers, peer-education, computer resources, campus policy, campus wide events. Other components of CADAPP targeted particular at-risk segments: free and confidential psychological 'drinker's checkup', list of drug/alcohol referral services available in the community, free psychological help for concerned family members and friends, alcohol self control training for on-campus fraternities


Outcomes
  • Frequency of use of 10 types of drugs including cannabis and cocaine
  • Past 30-day alcohol consumption
  • Perception of risk related to alcohol and other drugs use
  • Problems related to alcohol and other drug use


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaire

p. 746 "Impact of CADAPP was measured through anonymous surveys of students on each campus [..]"

Incomplete outcome data (attrition bias)
All outcomes
Low riskp. 751 "At baseline (fall) assessment, 1,400 surveys were distributed to enrolled UNM students, a sample of approximately 6% selected randomly by the university's computerized mailing list program. Of these, 567 surveys were returned and usable (41%). At the control campus, 1,080 surveys were distributed to a random sample of students, 457 of whom returned them (42.3%). [..] The return rates were 431 (31%) at UNM and 434 (34%) at NMSU"

Selective reporting (reporting bias)Unclear riskNo information regarding potential reporting bias

Newton 2010

MethodsStudy design: randomised controlled trial

Sampling: cluster sampling

Comparison group(s): other type of communication interventions (usual health classes)

Follow-up duration: 12 months

Study time span: March 2007 to November 2008 (21 months)


Participants764 13-year-old students from 10 Australian independent secondary schools (intervention branch: N = 397, 5 schools; control branch: N = 367, 5 schools). Students who enrol in independent schools come predominantly from high socioeconomic backgrounds


InterventionsClimate Schools course (Australia) is an Internet-based intervention founded on the social influence approach, derived from Bandura's social learning theory. The course delivered 2 sets of 6 40-minute lessons, each including 15 to 20-minute Internet-based lesson completed individually and 20 to 25-minute teacher-delivered activities. During the Internet-based part, students followed a cartoon storyline of teenagers experiencing real-life situations and problems with alcohol and cannabis


Outcomes
  • Use of alcohol (number of drinks per week) and cannabis (times per week)
  • Alcohol and cannabis knowledge
  • Alcohol and cannabis attitudes
  • Alcohol- and cannabis-related harms


NotesAssessment: baseline, immediately post, and 6 and 12 months following completion of the intervention

Hybrid intervention: both school- and Internet-based


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Low riskp. 750 "The 10 participating schools were assigned randomly using an online randomization system (www.randomizer.org) to either the control condition (usual drug education) or the intervention condition (the Climate Schools: Alcohol and Cannabis course)"

Allocation concealment (selection bias)Low riskAlthough full allocation concealment is not possible for this kind of study, there is low risk of selection bias because researchers administering the intervention were unlikely to know the children. See p. 750 "The Climate Schools: Alcohol and Cannabis course comprised the delivery of two sets of six 40-minute lessons. The Climate Schools: Alcohol module was delivered immediately after the baseline assessment, and the Climate Schools: Alcohol and Cannabis module was delivered 6 months later in the same school year. Each lesson included a 15–20-minute Internet-based lesson completed individually, where students followed a cartoon storyline of teenagers experiencing real-life situations and problems with alcohol and cannabis. The second part of each lesson was a predetermined activity delivered by the teacher to reinforce the information taught in the cartoons. Intervention group teachers were provided with a programme manual but no additional training."

Blinding of outcome assessment (detection bias)
All outcomes
Low riskp. 751 "A self-report questionnaire was completed online by all students in a classroom setting, where anonymity and confidentiality were assured"

Incomplete outcome data (attrition bias)
All outcomes
Low riskSimilar attrition % over the 2 study groups (figure 1, page 754)
p. 754 "Compared to students who were present at baseline and any follow-up occasion, students present only at baseline had significantly higher alcohol-related knowledge [7.66 versus 7.48 (of 16); F(1, 758) = 4.88, P < 0.05]. There were no significant differences on any other alcohol or cannabis outcome measures, nor was there evidence of differential attrition"

Selective reporting (reporting bias)Unclear riskNo study protocol was mentioned

Palmgreen 1991

MethodsStudy design: randomised controlled trial.

Sampling: random sampling (students were recruited from a variety of sources, including driver's licence listings, recruitment advertisements in local newspapers and shopper weekly, etc)

Comparison group(s): 2 experimental viewing conditions

  • one public service announcements (PSA) aimed at high sensation-seekers (HSSs)
  • one PSA aimed at low sensation-seekers (LSSs)


Follow-up duration: not applicable

Study time span: not specified, at least 1 day


Participants207 18- to 22-year-old youths


Interventions2 national-quality 30-second embedded PSAs, one aimed at HSS and the other at LSS (USA)


Outcomes
  • Attitude toward drug use
  • Intention to call a support hotline


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot clearly reported
p. 221 "LSSs ad HSSs were randomly assigned to one of the experimental conditions or the control group"

Allocation concealment (selection bias)Unclear riskNot reported
No baseline comparisons reported

Blinding of outcome assessment (detection bias)
All outcomes
Unclear riskBlinding of outcome assessors not reported

Incomplete outcome data (attrition bias)
All outcomes
Low riskNo missing data and clear reporting of sample size both of the intervention and control group

Selective reporting (reporting bias)Low riskNo study protocol available but clear reporting of main study hypothesis (p. 219) and outcomes measures

Palmgreen 2001

MethodsStudy design: interrupted time series study

Sampling: systematic sampling (geographical and grade stratification from enrolment lists of 7th to 10th graders in spring 1996)

Comparison group(s): pre-intervention surveys

Follow-up duration: n/a

Study time span: March 1996 to December 1998 (34 months)


Participants6371 youths from 7th to 10th grade (12- to 17-year-olds), 3174 from Fayette County and 3197 youths from Knox County


Interventions3 anti-marijuana public service announcements televised from January through April 1997 and from January through April 1998 in Fayette and Knox Counties (USA). These advertisements were based on the SENTAR (sensation-seeking targeting) prevention approach


Outcomes
  • Past 30-day use of marijuana


NotesThe 2 samples differed significantly on some independent (e.g. perceived peer and family drug use, delinquency) and dependent (use of marijuana) variables, although demographic and sensation-seeking variables were consistent between the 2 samples


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaire

p. 293 "Interviews were private and anonymous, with self-administration of drug and alcohol items via laptop computer"

Incomplete outcome data (attrition bias)
All outcomes
Unclear riskNot reported

Selective reporting (reporting bias)Unclear riskNo information regarding potential reporting bias

Polansky 1999

MethodsStudy design: randomised controlled trial

Sampling: systematic sampling (gender, classroom)

Comparison group(s): 2 × 2 × 4 design (replication × gender × treatment)

Follow-up duration: not specified

Study time span: not specified


Participants312 7th through to 9th graders from a rural south-western Mexican-American community


Interventions3 substance abuse prevention videotapes (USA) derived from different theoretical frameworks: information-based programming, social skills approach and assertiveness training (a subset of social skills approach)


Outcomes
  • Attitudes towards drugs
  • Use of drugs
  • Other: knowledge of videotape content and disposition to select socially appropriate responses


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskRandomisation procedure not clearly reported
p. 189 "…and then randomly assigned to one of the four treatment and control conditions"

Allocation concealment (selection bias)Unclear riskNot reported
No baseline comparisons reported

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaire

p. 191 "to permit collating pre-post protocols while preserving respondent anonymity, the students devised an identification code that they placed on all materials"

Incomplete outcome data (attrition bias)
All outcomes
Unclear riskUnclear reporting of the size of both the intervention and control group
Moreover unclear whether the final number of students (312) is the initial sample or is the final number of just those who answered (i.e. after drop-out)
Abstract "participants were 312 students"
p. 189 "153 seventh and eighth grade student responses and 159 ninth-grader responses were analysed"

Selective reporting (reporting bias)Low riskNo study protocol available but all outcome measures expected (per hypothesis p. 188) have been reported, including those not statistically significant (p. 192/194)

Scheier 2010

MethodsStudy design: prospective cohort study

Sampling: systematic sampling (representative of major racial groups)

Comparison group(s): lower exposure to intervention

Follow-up duration: 48 months

Study time span: April 1999 to March 2003 (48 months)


Participants2515 youth aged 12 to 18 interviewed by the National Survey of Parents and Youth (NSPY)


InterventionsThe National Youth Anti-Drug Media Campaign (USA), already described in Hornik 2006


Outcomes
  • Past 12-month episodes of drunkenness or cannabis intoxication
  • Past 30-day binge drinking (5 or more drinks in a row)
  • Past 30-day use of cigarettes


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous and computer-administered questionnaire

p. 248 "Assessment of alcohol and drug use relied on an Anonymous Computer Assisted Self-report Interview (ACASI)"

Incomplete outcome data (attrition bias)
All outcomes
Low riskSame as Hornik 2008 ("The overall  response rate among youths for the first round  was 65%, with 86% to 93% of still eligible  youths interviewed in subsequent rounds", page 2230 Evaluation of the National Youth Anti-Drug Media Campaign: 2004 Report of Findings, page 2-12, table 2-A "Completed interviews by wave")

Selective reporting (reporting bias)Unclear riskNot applicable

Schwinn 2010

MethodsStudy design: randomised controlled trial

Sampling: random sampling

Comparison group(s): no intervention

Follow-up duration: 6 months

Study time span: at least 8 months (not directly specified, but pretest was administered 6 weeks before intervention and last follow-up was assessed after 6 months)


Participants236 girls aged 13 to 14 from 42 US states and 4 Canadian provinces, recruited through the youth-oriented website Kiwibox.com™


InterventionsInternet-based gender-specific intervention (USA, Canada) composed by 12 sessions. This intervention is a pilot test of a gender-specific intervention based on the social learning theory and employs a social competence and skill building strategy. High interaction


Outcomes
  • Past 30-day alcohol, cigarette, marijuana, poly drug and total substance use
  • Mediator variables
    • Decision-making skills
    • Goal-setting skills
    • Drug resistance/refusal skills
    • Stress management
    • Social skills
    • Self esteem
    • Body esteem
    • Self efficacy


NotesBaseline and 6-month follow-up


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskp. 26 "After study enrolment, girls were randomly assigned to the intervention or control arm"

Allocation concealment (selection bias)Low riskAlthough full allocation concealment is not possible for this kind of study, there is low risk of selection bias because researchers administering the intervention were unlikely to know the children. See p. 26 "After completing online pretest measures, intervention girls were immediately directed to the first program session. Control girls were thanked for their time and reminded that they would be notified when the next survey was available."

Blinding of outcome assessment (detection bias)
All outcomes
Low riskComputer-administered questionnaire

p. 26 "After completing online pretest measures, intervention girls were immediately directed to the first program session. [..] Immediately following completion of the last program module, girls in the intervention group completed the post-test"

Incomplete outcome data (attrition bias)
All outcomes
Low riskp. 28 "Differential attrition was assessed across the three measurement occasions using the same variables analysed in baseline equivalency. Pretest to posttest attrition was 6.8%; the attrition rates for girls in intervention and control groups did not differ, X2 (1) = 1.74, p>0.05. At final follow-up, attrition was 9%; again, rates did not differ by study group, X2 (1) = 0.84, p>0.05"

Selective reporting (reporting bias)High riskProtocol not mentioned. Subjective outcomes were not described in full but only as predictors of objective outcomes (substance use)

Slater 2006

MethodsStudy design: quasi-randomised controlled trial (assignment to media condition was random; assignment to school condition was not fully random because of problem of staff scheduling in 7 of the 16 communities)

Sampling: randomised cluster sampling (treatment and control communities were extracted from 4 major regions of the US)

Comparison group(s): no intervention (8 intervention versus 8 non-intervention communities)

Follow-up duration: 24 months

Study time span: Autumn 1999 to Spring 2003 (˜42 months; but intervention lasted 24 months for each community, entry to the in project was different in different communities)


Participants4216 6th- and 7th-grade students; mean age at baseline was 12.2 years


InterventionsThe 'Be Under Your Own Influence' programme (USA) is a school- and community-based media effort on marijuana, alcohol and tobacco uptake. The programme emphasised "non-use as an expression of personal identity and the consistency of non-use with youth aspiration". The school-based intervention was research-based All Stars™ (13 sessions in the first year + 7 booster sessions in the second year); the community intervention was composed of workshops held by trained project staff


Outcomes
  • Lifetime and past 30-day use of marijuana
  • Lifetime and past 30-day episodes of alcohol intoxication
  • Lifetime and current smoking of cigarettes


NotesThis intervention ran concurrently with the Office of National Drug Policy's national anti-drug campaign (Hornik 2006; Scheier 2010), but their simultaneous effect was not assessed in this study


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskMatching procedure described but no specification of random sequence generation

Allocation concealment (selection bias)Low riskAlthough full allocation concealment is not possible for this kind of study, there is low risk of selection bias because researchers administering the intervention were unlikely to know the children

Blinding of outcome assessment (detection bias)
All outcomes
Unclear riskBlinding of outcome assessors not reported

Incomplete outcome data (attrition bias)
All outcomes
Low riskMissing data have been imputed using appropriate methods (p. 161)

Selective reporting (reporting bias)Low riskNo protocol available but the we do not suspect selective reporting bias

Slater 2011

MethodsStudy design: randomised controlled trial with a nested prospective cohort study

Sampling: systematic sampling (schools were recruited based on National Center for Educational Statistics district listings)

Comparison group(s): 4 groups, each including 10 schools and each comprising low to high exposure to the ONDCP campaign

  • Be Under Your Own Influence (BUYOI) intervention both at school and in the community
  • BUYOI intervention at school but not in the community
  • BUYOI intervention in the community but not at school
  • no BUYOI intervention neither at school nor in the community


Follow-up duration: 24 months

Study time span: Autumn 2005 to Spring 2009 (˜42 months)


Participants3236 students, mean age 12.4 ± 0.6 years


InterventionsThe Office of National Drug Control Policy's (ONDCP) 'Above the Influence' media campaign (USA) and a school- and community-based mass media intervention, 'Be Under Your Own Influence' (BUYOI; USA). They both started in 2005 and ran concurrently

  • The ONDCP's campaign is the rebranded version of the national anti-drug campaign launched in 1998 (Hornik 2006; Palmgreen 2007; Scheier 2010). This version, like the original one, used televised ads supplemented by printed ads (e.g. posters)
  • The BUYOI campaign is a replication and extension of a campaign launched in 1999 (Slater 2006). This campaign employed only printed ads and was implemented both in schools and communities


Although the ONDCP's campaign used far more creative executions given its funding levels, both campaigns were similar in concept, i.e. both linked substance use with autonomy and aspiration threats


Outcomes
  • Attitudes: autonomy and aspiration inconsistent with marijuana use
  • Lifetime, past 90-day and past 30-day use of marijuana


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Low riskPage 15 "Random assignment used a group-matching procedure: NCES data on community demographics and location were used to generate possible randomization schemes in which major demographics and location were balanced to the degree possible across experimental conditions and one of the acceptable schemes was randomly selected."

Allocation concealment (selection bias)Low riskAlthough full allocation concealment is not possible for this kind of study, there is low risk of selection bias because researchers administering the intervention were unlikely to know the children

Blinding of outcome assessment (detection bias)
All outcomes
Unclear riskIt is unclear who administered the questionnaires and whether they were anonymous

Incomplete outcome data (attrition bias)
All outcomes
Low riskReasons for missing outcome data unlikely to be related to true outcome

Selective reporting (reporting bias)Low riskNo protocol available but the we do not suspect selective reporting

Wyoming Meth 2011

MethodsStudy design: interrupted time series study

Sampling: 4-stage probability sampling

Comparison group(s): pre-intervention survey

Follow-up duration: n/a

Study time span: April 2008 to May 2011 (34 months)


Participants5700 youths (909 + 913 + 2652 + 1226)


InterventionsMeth Project (USA), a "messaging campaign, supported by community outreach, and public policy initiatives". The campaign comprises "television, radio, print, billboard, and Internet advertising"


Outcomes
  • Past-month use of methamphetamine
  • Attitudes towards methamphetamine and other drugs
  • Perceptions concerning methamphetamine and other drugs
  • Informations sources and advertising awareness
  • Statewide Meth Project awareness and perceptions


Notes


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskNot applicable

Allocation concealment (selection bias)Unclear riskNot applicable

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaires

Incomplete outcome data (attrition bias)
All outcomes
Unclear riskNot reported

Selective reporting (reporting bias)Unclear riskNo information regarding potential reporting bias

Yzer 2003

MethodsStudy design: randomised controlled trial

Sampling: random sampling (from middle and high schools)

Comparison group(s):

  • no intervention (documentary with no advertisements)
  • gateway condition (explicit: 4 anti-hard drug followed by a teenage girl's testimonial about how her trial use of marijuana led to using hard drugs)
  • implicit gateway condition (2 anti-marijuana and 2 anti-hard drugs advertisements without explicit reference to the gateway concept)
  • hard drugs condition (same advertisements of gateway condition, but not followed by testimonials)


Follow-up duration: not applicable (post-only design)

Study time span: March 2000 to not specified


Participants418 students of middle/high schools in urban Philadelphia, mean age 14 ± 1.89 years


InterventionsAnti-marijuana and anti-hard drugs advertisements embedded in a documentary video (USA)


Outcomes
  • Intention to use marijuana in the next 12 months
  • Attitude towards marijuana
  • Perceptions about marijuana


NotesSimilar to Zhao 2006, many of the authors wrote both papers


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Low riskParticipants were randomly assigned to 1 of the 4 experimental conditions, and the stimuli were randomly presented using a randomisation feature in MediaLab software. (Personal communication with the author)

Allocation concealment (selection bias)Low riskParticipants did not know which condition they were assigned to, and thus did not know which stimuli they and participants in other conditions were exposed to. (Personal communication with the author)

Blinding of outcome assessment (detection bias)
All outcomes
Low riskAnonymous questionnaire

p. 135 "All videos and the questionnaire were programmed onto a laptop computer using an interactive program that allows random ordering of questions and videos within blocks"

Incomplete outcome data (attrition bias)
All outcomes
Low riskThere were no missing data. (Personal communication with the author)

Selective reporting (reporting bias)Low riskNo protocol available but the we do not suspect selective reporting bias

Zhao 2006

MethodsStudy design: randomised controlled trial

Sampling: not specified (informational letters to parents in the 2 school-based studies, mall-intercept of lists held by market researchers for the mall-based study)

Comparison group(s): no intervention (documentary about television production, without the embedded anti-marijuana advertisements)

Follow-up duration: not applicable (post-only design)

Study time span: not specified


Participants435 youths whose mean age was 15.2 ± 1.88 years


Interventions3 anti-marijuana advertisements (USA) addressing normative beliefs. The advertisements were embedded and randomly included in a video documentary about television production


Outcomes
  • Behavioural beliefs towards marijuana (perceptions)
  • Intention to use marijuana
  • Social norms on marijuana (perceptions)


NotesResults were based on combined data from 3 studies done at different points in time, but "identical in terms of methodology, procedures, experimental conditions, ad the structure of the outcome questionnaire". However, whereas study 1 and 2 were collected at middle and high schools, study 3 was conducted at various malls around the country


Risk of bias

BiasAuthors' judgementSupport for judgement

Random sequence generation (selection bias)Unclear riskp. 190 "Participants were randomly assigned to condition", but randomisation details are not reported

Allocation concealment (selection bias)Low riskAlthough full allocation concealment is not possible for this kind of study, there is low risk of selection bias because researchers administering the intervention were unlikely to know the children. See p. 190: "The experimental group saw the three advertisements that challenged undesirable normative beliefs about marijuana use (see Table 1 for a description of the messages). The advertisements were embedded and randomly rotated in a video documentary about television production. The control group was not exposed to any anti-marijuana messages but saw the same documentary as the experimental group"

Blinding of outcome assessment (detection bias)
All outcomes
Low riskComputer-administered questionnaire

p. 191 "The instrument (including the video clips) was programmed onto laptop computers using an interactive program called MediaLab (Jarvis, 1998), which allows random ordering of blocks of questions and videos within the questionnaire"

Incomplete outcome data (attrition bias)
All outcomes
Low riskAll participants completed the test (being a post-only design); p. 190 "All three studies used the same between-subjects, post-only design, with one experimental condition and one control condition"

Selective reporting (reporting bias)Unclear riskNo study protocol was mentioned

 
Characteristics of excluded studies [ordered by study ID]

StudyReason for exclusion

Alemi 1996Target population is pregnant women who already use or used cocaine

An 2007This intervention aims to promote inquiry of prescription medicines/treatments, not to hinder use of illicit drugs

Andrews 1995The purpose of the campaign was to promote public awareness of the link between addiction and child maltreatment, not to prevent addiction

Barber 1990Target population mean age is 40

Beaudoin 2007Presented outcomes are not included among those of this review

Beck 2008Overview of drugs prevalence and school-based prevention interventions in France. Some information about therapeutic interventions, but no information about mass media prevention interventions

Belenko 2009This study analyses data from the National Survey of Parents and Youth, which was not designed to provide quality information about exposure to anti- or pro-drug websites. This study aims to find factors (e.g. gender, parent-reported income, prior exposure to drugs) associated with viewing of drug websites, not to assess whether viewing of anti-drug websites can influence outcomes included in the protocol of this review

Black 1994This study aims to assess consistency of data collected with 2 different sampling methods

Brannon 1989None of the evaluated outcomes (i.e. participation, satisfaction and perceived efficacy of programme) met the inclusion criteria for this review

Chambers 2005Not a mass media intervention

Chiauzzi 2008This study assesses the effectiveness of an online stress management tool. Outcomes do not include substance use, intention to use or any other outcome relevant to this review

Collins 1991This paper aims to prevent alcohol abuse

Cook 1999Review of books and media, not of studies

David 2006Evaluated intervention is adolescent discussion about anti-drug advertisements, not advertisements themselves

DeJong 1999This paper raises concern about the Office of National Drug Control Policy (ONDCP)'s National Youth Anti-Drug Media Campaign without reporting results of its effectiveness assessment

Di Noia 2003The majority of recruited professionals were older than 26 and the assessed outcomes are not among those needed for inclusion

Donohew 2000The aim of the study is to understand the relationship between mediators (sensation-seeking and decision-making processes) and alcohol and risky sexual behaviours in adolescents

Epstein 1999Survey with control group but without pre-intervention questionnaire

Erceg-Hurn 2008It is not possible to compare different years due to the different methodology used in surveys (see also commentary paper Erceg-Hurn 2008)

Everett 1995This study does not evaluate intervention effectiveness but matching between HSV/LSV interventions and HSV/LSV subjects

Flay 2000Reviews of mainly anti-tobacco media-, school- and community-based interventions

Hannon 2000Narrative review of key African American community values and provides recommendations as to how this information might be incorporated into the development of anti-drug messages and materials targeted at African Americans

Harrington 2003This study does not evaluate intervention effectiveness but matching between HSV/LSV interventions and HSV/LSV subjects

Helme 2007Intervention was an anti-smoking campaign

Johnson 1990The mass media intervention was administered to both study groups

Jordan 2005This study design (survey) does not allow us to evaluate intervention effectiveness

Kang 2009This study is an evaluation of the perceived effectiveness of specific elements of the interventions, not the effectiveness of whole interventions on outcome variables included in the protocol for this review

Know the Score 2007For the 2 cocaine reports: the 4 study waves differed slightly but in many respects (age and working status of respondents, survey locations and, more importantly, survey questions)

For the 2 heroin reports: respondent age is not fully comparable across study waves. Additionally, participants in waves 1 and 3 were older than 25

Lorch 1994No pretest drug-related measure was taken. This study aims to predict responses to PSA and drug use by different sensation-seeking profiles

Lubman 2007Narrative review on substance addiction prevention. Data were not presented here

Marsiglia 2009This study evaluates a school-based intervention which has no media-related component

Myers 2006Not a prevention intervention. It does not include illicit drug-related outcomes

Palmgreen 2007This study does not evaluate intervention effectiveness but matching between the intervention and HSV/LSV subjects

Pentz 1990The fffect of the mass media component could not be disentangled from other components

Ramirez 1999Description of theoretical basis, development and implementation of 'Mirame!/Look at Me!' media- and school-based programme for substance abuse among Hispanic youth. However, the programme's effectiveness was not assessed

Reis 1994Survey. This study design does not allow us to evaluate intervention effectiveness

Ruggiero 2006Participants are older than 26

Schmeling 1980Intervention targets prescription drug abusers

Siegel 2008No blank control, one group focusing on physical harms of inhalant use, the other focusing on social harms

Skinner 1995aThe outcome (perceived persuasiveness) is not among the outcome measures included in our protocol

Sloboda 2006This book does not include data on studies evaluating mass media programmes

Spitzer 2010Outcomes concern 'values' and therefore do not meet the inclusion criteria

Stephenson 2002The aim of this study was to find predictors of exposure from an anti-marijuana media campaign, not to evaluate the effectiveness of the campaign

Stephenson 2002aCBA study aiming to link perceived message sensation value and viewer's reaction to an anti-heroin PSA

Stephenson 2003Survey with control group aiming to evaluate sensation-seeking as a moderating variable

Stephenson 2005This study analyses the content of ads but does not assess their effectiveness

Stevens 1996School-based intervention with added community activities

Stryker 2003Ecological study about the impact of media coverage of the negative consequences of marijuana use. This study does not assess the effectiveness of a single prevention intervention

Sussman 1987Survey with a control group but without a pre-intervention questionnaire

Tait 2010Systematic review on Internet-based interventions for the treatment of alcohol misuse

Taylor 1984Outcomes in the pilot study (the statewide intervention was not evaluated) were knowledge, attitudes and behaviours about friendships and human relationships, not substance use/misuse

Varshavsky 2003Qualitative content analysis of a national campaign

Werch 2010Not a mass media intervention

 
Characteristics of studies awaiting assessment [ordered by study ID]
Block 2002

Methods

Participants

Interventions

Outcomes

NotesWe contacted authors for results and are waiting for a response

Duncan 2000

Methods

Participants

Interventions

Outcomes

NotesWe contacted authors for results and are waiting for a response

Flay 1986

Methods

Participants

Interventions

Outcomes

NotesWe were unable to retrieve the paper's full text

Longshore 2006

Methods

Participants

Interventions

Outcomes

NotesWe contacted authors for results and are waiting for a response

Marsch 2007

Methods

Participants

Interventions

Outcomes

NotesWe were unable to retrieve the paper's full text

Moore 2011

Methods

Participants

Interventions

Outcomes

NotesWe were unable to retrieve the paper's full text

Moreno 2009

Methods

Participants

Interventions

Outcomes

NotesWe contacted authors for results and are waiting for a response

Skinner 1995

Methods

Participants

Interventions

Outcomes

NotesWe were unable to retrieve the paper's full text

Williams 2005

Methods

Participants

Interventions

Outcomes

NotesWe contacted authors for results and are waiting for a response

 
Comparison 1. Mass media versus no mass media intervention (RCT)

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Drug use55470Std. Mean Difference (IV, Random, 95% CI)-0.02 [-0.15, 0.12]

 2 Intention to use drugs41270Std. Mean Difference (IV, Fixed, 95% CI)-0.07 [-0.19, 0.04]

 
Table 1. 'Risk of bias' assessment of interrupted time series studies

Miller 2000

CriterionScoreNotes

a) Protection against secular changes

The intervention is independent of other changesDone"The usual environmental influences such as prices, taxes, state regulations, campus policies, and enforcement did not change substantially during the study period. Neither was there any reason to expect that students on the two campuses would respond differentially to anonymous surveys. The only obvious difference between the two campuses that might be expected to affect substance use differentially was the implementation of the prevention program at UNM", page 756

There are sufficient data points to enable reliable statistical inferenceNot done2 data points (before and after)

Formal test for trend. Complete this section if authors have used ANOVA modellingDone

b) Protection against detection bias

Intervention unlikely to affect data collectionDone"All questionnaires were completed anonymously. To encourage participation, those who returned the survey (by mail) were entered into a lottery for cash prizes by separating a numbered ticket, returning one part with the completed survey and retaining the other half. Winning numbers were announced through the campus newspaper, the Daily Lobo. As an additional incentive for the follow-up survey, respondents were invited to participate in a contest to guess the actual levels of alcohol/drug use on campus, as revealed by the first survey", page 750

Blinded assessment of primary outcome(s)DoneAnonymous surveys, page 750

c) Completeness of data setDone"At baseline (fall) assessment, 1,400 surveys were distributed to enrolled UNM students, a sample of approximately 6% selected randomly by the university s computerized mailing list program. Of these, 567 surveys were returned and usable (41%). At the control campus, 1,080 surveys were distributed to a random sample of students, 457 of whom returned them (42.3%). [..] The return rates were 431 (31%) at UNM and 434 (34%) at NMSU", page 751

d) Reliable primary outcome measure(s)Done"Use measures (14 items) included a frequency (number of drinking days per 30) and quantity index of drinking (number of standard drinks consumed per drinking occasion; range: 0-15) that were multiplied to form a single quantity frequency measure (number of drinks per month) [..]", page 750

"Problem measures included 14 indicators of alcohol dependence and adverse consequences of heavy drinking or illicit drug use in the prior year. [..]", page 750

"Risk assessment included 13 items regarding the extent to which students perceived risk or consequences related to alcohol or other drug use [..]", page 750


Palmgreen 2001 (includes Stephenson 1999)

CriterionScoreNotes

a) Protection against secular changes  

The intervention is independent of other changesUnclear

There are sufficient data points to enable reliable statistical inferenceDone32 data points

Formal test for trend. Complete this section if authors have used ANOVA modellingDoneANOVA modelling was used. See from page 186 on

b) Protection against detection bias  

Intervention unlikely to affect data collectionDoneMethodology of data collection is not reported to have changed across data points

Blinded assessment of primary outcome(s)DoneAnonymous computer-administered questionnaire (p. 293)

c) Completeness of data setUnclear 

d) Reliable primary outcome measure(s)Done30-day use of marijuana, attitudes, beliefs, intentions


Idaho Meth 2010, Colorado Meth 2011, Georgia Meth 2011, Hawaii Meth 2011 and Wyoming Meth 2011

CriterionScoreNotes

a) Protection against secular changes  

The intervention is independent of other changesUnclear

There are sufficient data points to enable reliable statistical inferenceDoneData points for each study ranged from 2 to 4 including only one baseline survey. However, overall, there are a sufficient number of observations

Formal test for trend. Complete this section if authors have used ANOVA modellingNot done

b) Protection against detection bias

Intervention unlikely to affect data collectionDoneDespite some slight changes, methodology of data collection is consistent across studies and across data points

Blinded assessment of primary outcome(s)DoneAnonymous questionnaires

c) Completeness of data setUnclearNot applicable

d) Reliable primary outcome measure(s)DonePast-month use of marijuana, attitudes, perceptions


Carpenter 2011

CriterionScoreNotes

a) Protection against secular changes  

The intervention is independent of other changesDoneAdjustment by many individual and market variables (page 949)

There are sufficient data points to enable reliable statistical inferenceNot done3 data points (page 949)

Formal test for trend. Complete this section if authors have used ANOVA modellingDone"multivariate logistic regression" (page 949)

b) Protection against detection bias

Intervention unlikely to affect data collectionDoneAds were broadcasted independently on the surveys

Blinded assessment of primary outcome(s)DoneMonitoring the Future (MTF) surveys used anonymous questionnaires

c) Completeness of data setUnclear

d) Reliable primary outcome measure(s)DonePast-month and lifetime marijuana use (page 951)

 
Table 2. 'Risk of bias' assessment of cohort studies (Hornik 2006, Scheier 2010, Slater 2011)

Hornik 2006

CriterionScore/InfoNotes

In a well-conducted cohort study:  

The study addresses an appropriate and clearly focused questionWell covered"We examined the cognitive and behavioral effects of the National Youth Anti-Drug Media Campaign on youths aged 12.5 to 18 years and report core evaluation results", abstract

Selection of subjects  

The 2 groups being studied are selected from source populations that are comparable in all respects other than the factor under investigationWell covered"The sample was selected to provide an efficient and nearly unbiased cross-section of US youths and their parents. Respondents were selected through a stratified 4-stage probability sample design: 90 primary sampling units—typically county size—were selected at the first stage, geographical segments were selected within the sampled primary sampling units at the second stage, households were selected within the sampled segments at the third stage, and then, at the final stage, 1 or 2 youths were selected within each sampled household, as well as 1 parent in that household.", page 2229-30

The study indicates how many of the people asked to take part did so, in each of the groups being studiedWell coveredEvaluation of the National Youth Anti-Drug Media Campaign: 2004 Report of Findings, Appendix A, page A-6, table A-1 and page A-11 tables A-8 to A-10

The likelihood that some eligible subjects might have the outcome at the time of enrolment is assessed and taken into account in the analysisWell covered"Analyses were restricted to youths who were nonusers of marijuana at the current round (for cross-sectional analyses) or at the previous round (for lagged analyses).", page 2232

What percentage of individuals or clusters recruited into each arm of the study dropped out before the study was completed35%"The overall response rate among youths for the first round was 65%, with 86% to 93% of still eligible youths interviewed in subsequent rounds.", page 2230

Evaluation of the National Youth Anti-Drug Media Campaign: 2004 Report of Findings, page 2-12, table 2-A "Completed interviews by wave"

Comparison is made between full participants and those lost to follow-up, by exposure statusNot reported

Assessment  

The outcomes are clearly definedWell covered"For 3 reasons, all drug-related measures reported here relate to marijuana use. [..] Four measures or indices represented the following constructs: (1) marijuana intentions, (2) marijuana beliefs and attitudes, (3) social norms, and (4) self-efficacy to resist use.", page 2230

The assessment of outcome is made blind to exposure statusNot applicableBlinding to exposure status was not applicable for this study

Where blinding was not possible, there is some recognition that knowledge of exposure status could have influenced the assessment of outcomeWell covered"A measure of general exposure to antidrug advertising was derived from responses to questions about advertising recall for each medium or media grouping: television and radio, print, movie theatres or videos, and outdoor advertising.", page 2230

The measure of assessment of exposure is reliableWell covered"For 3 reasons, all drug-related measures reported here relate to marijuana use.", page 2230

Evidence from other sources is used to demonstrate that the method of outcome assessment is valid and reliableWell covered"For 3 reasons, all drug-related measures reported here relate to marijuana use. First, marijuana is by far the illicit drug most heavily used by youths. Second, for other drugs, the low levels of use meant that the NSPY sample sizes were not large enough to detect meaningful changes in use with adequate power. Third, to the extent that the campaign did target a specific drug, it was almost always marijuana. [..] The cognitive measures were developed on the basis of 2 health behavior theories, the theory of reasoned action and social cognitive theory", page 2230

Exposure level or prognostic factor is assessed more than onceWell covered"3 nationally representative cohorts of US youths aged 9 to 18 years were surveyed at home 4 times.", abstract

Confounding  

The main potential confounders are identified and taken into account in the design and analysisWell covered"Potential confounder measures. The analyses employed propensity scoring for confounder control by weighting adjustments,9–14 incorporating a wide range of standard demographic variables and variables known to be related to youths' drug use or thought likely to be related to exposure to antidrug messages. Propensity scores were developed for the general and specific exposure measures. More than 150 variables were considered possible confounders.", page 2231

Statistical analysis  

Have confidence intervals been provided?Well coveredTables 1-4, pages 2233-4

Overall assessment of the study  

How well was the study done to minimise the risk of bias or confounding, and to establish a causal relationship between exposure and effect?

Code ++,+, or −
++Propensity scoring from 150 confounders, page 2231

Taking into account clinical considerations, your evaluation of the methodology used, and the statistical power of the study, are you certain that the overall effect is due to the exposure being investigated?YesThis study includes very good control for possible confounders

 

 

Are the results of this study directly applicable to the patient group targeted in this guideline?UnclearResults are applicable to US youth; it is unclear whether they are generalisable outside the US

Description of the study  

Do we know who the study was funded by?Public Funds (NIDA), Government (Congress)"Research for and preparation of this article were supported by the National Institute on Drug Abuse (grants 3-N01-DA085063-002 and 1-R03-DA-020893-01). The evaluation of the National Youth Anti-Drug Media Campaign was funded by Congress as part of the original appropriation for the campaign. The White House Office of National Drug Control Policy directly supervised the campaign. The National Institute on Drug Abuse supervised the evaluation; Westat, with the Annenberg School for Communication at the University of Pennsylvania as a subcontractor, received the contract. All authors were funded for this evaluation and other projects by the National Institute on Drug Abuse.", page 2235

How many centres are patients recruited from?USA as a whole"90 primary sampling units—typically county size—were selected at the first stage, geographical segments were selected within the sampled primary sampling units at the second stage, households were selected within the sampled segments at the third stage, and then, at the final stage, 1 or 2 youths were selected within each sampled household, as well as 1 parent in that household.", page 2230

From which countries are patients selected? (Select all those involved. Note additional countries after 'Other')USA 

What is the social setting (i.e. type of environment in which they live) of patients in the study?Mixed"More than 150 variables were considered possible confounders. [..] They include [..] urban–rural residency; [..]", page 2231

What criteria are used to decide who should be INCLUDED in the study?4-stage selection"Respondents were selected through a stratified 4-stage probability sample design: 90 primary sampling units—typically county size—were selected at the first stage, geographical segments were selected within the sampled primary sampling units at the second stage, households were selected within the sampled segments at the third stage, and then, at the final stage, 1 or 2 youths were selected within each sampled household, as well as 1 parent in that household.", page 2229-30

What criteria are used to decide who should be EXCLUDED from the study?Youth living in boarding schools and college dormitories"As mentioned previously, youth residing in group quarters were not eligible for selection in any of the three recruitment waves. Thus, youth living in boarding schools and college dormitories were excluded from the scope of the survey. This exclusion was made because it was felt that dormitory residents could not be easily interviewed at their parents' homes and that their experiences were so", Report, A-10

What intervention or risk factor is investigated in the study? (Include dosage where appropriate)The National Youth Anti-Drug Media Campaign 

What comparisons are made in the study (i.e. what alternative treatments are used to compare the intervention/exposure with). Include dosage where appropriateLower exposure versus higher exposure to anti-drug campaign"The analyses reported here were based on 3 types of measures: recalled exposure to antidrug messages aired by the campaign and other sources; cognitions and behavior related to marijuana, as outcomes; and individual and household characteristics, including a wide range of variables known to be related to drug cognitions and use and to exposure to antidrug messages.", page 2230

What methods were used to randomise patients, blind patients or investigators, and to conceal the randomisation process from investigators?Randomisation: not applicable, but propensity scoring was employed

Blinding of patients: not applicable
Blinding of investigators: not reported
Randomisation concealment: not applicable
 

How long did the active phase of the study last?September 1999 to June 2004 (58 months) 

How long were patients followed up for, during and after the study?November 1999 to June 2004 (56 months). 

List the key characteristics of the patient population. Note if there are any significant differences between different arms of the trialRepresentative of US youths aged 9 to 18"The sample was selected to provide an efficient and nearly unbiased cross-section of US youths and their parents", page 2229

Record the basic data for each arm of the study. If there are more than 4 arms, note data for subsequent arms at the bottom of the page Tables 1-4, pages 2233-4

Record the basic data for each IMPORTANT outcome in the study. If there are more than 4, note data for additional outcomes at the bottom of the page Tables 1-4, pages 2233-4

Notes. Summarise the authors' conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your questionThrough June 2004, the campaign is unlikely to have had favourable effects on youths and may have had delayed unfavourable effects

The evaluation challenges the usefulness of the campaign
 


Scheier 2010

CriterionScore/InfoNotes

In a well-conducted cohort study:  

The study addresses an appropriate and clearly focused questionWell covered"In this study, we examined whether awareness (recall) of the National Youth Anti-Drug Media Campaign (NYADMC) benefited youth by attenuating their drug use.", abstract

Selection of subjects  

The 2 groups being studied are selected from source populations that are comparable in all respects other than the factor under investigationWell coveredSame as Hornik 2008 ("The sample was selected to provide an efficient and nearly unbiased cross-section of US youths and their parents. Respondents were selected through a stratified 4-stage probability sample design: 90 primary sampling units—typically county size—were selected at the first stage, geographical segments were selected within the sampled primary sampling units at the second stage, households were selected within the sampled segments at the third stage, and then, at the final stage, 1 or 2 youths were selected within each sampled household, as well as 1 parent in that household.", page 2229-30)

The study indicates how many of the people asked to take part did so, in each of the groups being studiedWell coveredSame as Hornik 2008 (Evaluation of the National Youth Anti-Drug Media Campaign: 2004 Report of Findings, Appendix A, page A-6, table A-1 and page A-11 tables A-8 to A-10)

The likelihood that some eligible subjects might have the outcome at the time of enrolment is assessed and taken into account in the analysisWell coveredSame questionnaire was administered at baseline and at follow-up. "National Survey of Parents and Youth (NSPY) [..] could be used to assess youths' awareness of the campaign messages and monitor any corresponding changes in drug use trends.", page 241-2

What percentage of individuals or clusters recruited into each arm of the study dropped out before the study was completed35%Same as Hornik 2008 ("The overall  response rate among youths for the first round

 was 65%, with 86% to 93% of still eligible youths interviewed in subsequent rounds.

", page 2230 Evaluation of the National Youth Anti-Drug Media Campaign: 2004 Report of Findings, page 2-12, table 2-A "Completed interviews by wave")

Comparison is made between full participants and those lost to follow-up, by exposure statusWell covered"Attrition analyses were structured to determine whether certain factors operate  systematically to cause dropout from the study. Proportional analyses using the v2 test were used for cross tabulation of binary measures and logistic regression modelling to examine the optimal predictors of retention (coded '1' stay and '0' dropout). We used the WesVar software program to estimate logistic regression models of  panel attrition. This statistical modelling program enables us to adjust (through poststratification) the sample variance estimators for the undersampling of primary  sampling units and correct any bias in parameter estimates related directly to the complex sampling design (using replicate variance estimators to adjust standard errors for design effects).

Proportional tests indicated that panel youth were significantly more likely to be female, smoke more cigarettes, drink alcohol, and smoke marijuana (all v2 proportional tests significant at the p .0001) compared with dropout youth. Given the large number of variables possibly related to retention status, logistic models were run separately for five individual domains (demographics, campaign awareness, drug use, school-related factors, and psychosocial risk).7 Following tests of the individual domains, we culled only significant predictors and tested these in a combined model predicting retention. The final model indicated that retained youth were less at risk for marijuana use (unstandardized b =-3.51, p<= .0001, OR =.03), engaged in more antisocial behavior (evidencing suppression: [b = .23, p <=.0001, OR = 1.26]), spent fewer hours listening to the radio on a daily basis (b =-.09, p <=.01, OR = .91), and were more likely to have attended school in the past year (b = 1.05, p<= .01, OR = 2.87) compared with their dropout counterparts. Using the Cox-Snell likelihood pseudo-R2 statistic, the model accounted for 12% of the variance in retention status, F(14,87) = 12.127, p <=.0001.",

page 250

Assessment  

The outcomes are clearly definedWell covered"Assessment of alcohol and drug use relied on an Anonymous Computer Assisted Self-report Interview (ACASI). Two alcohol use items6 assessed being drunk or high ("How many times were you drunk or very high from alcohol in the last 12 months?") with response categories ranging from "I don't use alcohol" (0) through "40 or more occasions" (7); and heavy alcohol use based on a measure of binge drinking ("How many days have you had five or more drinks in the last 30 days?") with response categories ranging from "I don't drink" (0) through "10 or more times" (6). Cigarette use was assessed with a single item ("How many cigarettes smoked a day during the last 30 days?") with response categories ranging from "None" (0) through "More than 35 per day, about 2 packs or more" (7). A single frequency item assessed marijuana involvement ("How many times have you used marijuana in the last 12 months?") with response categories ranging from "I have never used marijuana" (0) through "40 or more occasions" (6).", page 248

The assessment of outcome is made blind to exposure statusNot applicableBlinding to exposure status was not applicable for this study

Where blinding was not possible, there is some recognition that knowledge of exposure status could have influenced the assessment of outcomeNot reported 

The measure of assessment of exposure is reliableWell covered"Turning to the campaign awareness parameters, we see two findings worth noting. First, growth in campaign awareness is positive for the earlier years (12 to 14), except for television viewing behavior, which had a slope not significantly different from zero. As these youth became older (14 to 18), their awareness declined for every media venue except specific recall (videos shown on laptops) and radio listening behavior. Also, the magnitude of the slope terms were considerably larger at the younger age for recall of stories about drugs and youth, brand awareness, specific recall, and radio listening but larger in magnitude for television (declining) as these youth transitioned to high school.", page 253

"Figure 2 graphically presents a generic template for testing the bivariate cohort growth models. Again, two slope trends are posited to capture the different rates of growth for youth when they were younger versus when they were older, and this is repeated for both drug use (D) and awareness (A) measures.", page 253

Evidence from other sources is used to demonstrate that the method of outcome assessment is valid and reliableWell covered"Assessment of alcohol and drug use relied on an Anonymous Computer Assisted Self-report Interview (ACASI). Two alcohol use items6 assessed being drunk or high  ("How many times were you drunk or very high from alcohol in the last 12 months?") with response categories ranging from "I don't use alcohol" (0) through  "40 or more occasions" (7); and heavy alcohol use based on a measure of binge drinking ("How many days have you had five or more drinks in the last 30 days?")  with response categories ranging from "I don't drink" (0) through "10 or more times" (6). Cigarette use was assessed with a single item ("How many cigarettes smoked a day during the last 30 days?") with response categories ranging from "None" (0) through "More than 35 per day, about 2 packs or more" (7). A single frequency item assessed marijuana involvement ("How many times have you used marijuana in the last 12 months?") with response categories ranging from "I have never used marijuana" (0) through "40 or more occasions" (6).", page 248

Exposure level or prognostic factor is assessed more than onceWell coveredYes: 4 rounds of data collection. Table 1, page 249

Confounding  

The main potential confounders are identified and taken into account in the design and analysisNot reported 

Statistical analysis  

Have confidence intervals been provided?NoPage 264

Overall assessment of the study  

How well was the study done to minimise the risk of bias or confounding, and to establish a causal relationship between exposure and effect?

Code ++, +, or −
-"...there was no "intervention" to speak of, but rather the campaign took shape as a naturalistic observational study conducted at a particular point in time with no clear demarcation from various historical influences that could affect patterns of reported drug use", page 264

Taking into account clinical considerations, your evaluation of the methodology used, and the statistical power of the study, are you certain that the overall effect is due to the exposure being investigated?No, because no adjustment for confounders was reported 

Are the results of this study directly applicable to the patient group targeted in this guideline?UnclearResults are applicable to US youth; it is unclear whether they are generalisable outside the US

Description of the study  

Do we know who the study was funded by?No 

How many centres are patients recruited from?USA as a wholeSame as Hornik 2008 ("90 primary sampling units—typically county size—were selected at the first stage, geographical segments were selected within the sampled primary sampling units at the second stage, households were selected within the sampled segments at the third stage, and then, at the final stage, 1 or 2 youths were selected within each sampled household, as well as 1 parent in that household.", page 2230)

From which countries are patients selected? (Select all those involved. Note additional countries after 'Other')USA 

What is the social setting (i.e. type of environment in which they live) of patients in the study?MixedSame as Hornik 2008 ("More than 150 variables were considered possible confounders. [..] They include [..] urban–rural residency; [..]", page 2231)

What criteria are used to decide who should be INCLUDED in the study?4-stage selectionSame as Hornik 2008 ("Respondents were selected through a stratified 4-stage probability sample design: 90 primary sampling units—typically county size—were selected at the first stage, geographical segments were selected within the sampled primary sampling units at the second stage, households were selected within the sampled segments at the third stage, and then, at the final stage, 1 or 2 youths were selected within each sampled household, as well as 1 parent in that household.", page 2229-30)

What criteria are used to decide who should be EXCLUDED from the study?Youth living in boarding schools and college dormitoriesSame as Hornik 2008 ("As mentioned previously, youth residing in group quarters were not eligible for selection in any of the three recruitment waves. Thus, youth living in boarding schools and college dormitories were excluded from the scope of the survey. This exclusion was made because it was felt that dormitory residents could not be easily interviewed at their parents' homes and that their experiences were so", Report, Appendix A, A-10)

What intervention or risk factor is investigated in the study? (Include dosage where appropriate)The National Youth Anti-Drug Media Campaign"...there was no "intervention" to speak of, but rather the campaign took shape as a naturalistic observational study conducted at a particular point in time with no clear demarcation from various historical influences that could affect patterns of reported drug use", page 264

What comparisons are made in the study (i.e. what alternative treatments are used to compare the intervention/exposure with). Include dosage where appropriateExposure versus drug useVarius models, e.g. see page 256

What methods were used to randomise patients, blind patients or investigators, and to conceal the randomisation process from investigators?Randomisation: not applicable
Blinding of patients: not applicable
Blinding of investigators: not reported
Randomisation concealment: not applicable
 

How long did the active phase of the study last?September 1999 to June 2004 (58 months) 

How long were patients followed up for, during and after the study?November 1999 to June 2004 (56 months) 

List the key characteristics of the patient population. Note if there are any significant differences between different arms of the trialRepresentative of US youths aged 9 to 18Same as Hornik 2008 ("The sample was selected to provide an efficient and nearly unbiased cross-section of US youths and their parents", page 2229)

Record the basic data for each arm of the study. If there are more than 4 arms, note data for subsequent arms at the bottom of the pageTable 2, page 251 

Record the basic data for each IMPORTANT outcome in the study. If there are more than 4, note data for additional outcomes at the bottom of the pageTables 3-4-5, pages 252-7 

Notes. Summarise the authors' conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your questionWhen they were younger, these youth accelerated their drug use and reported increasing amounts of campaign awareness. When they were older, [..] no effects for marijuana were significant but trended in the direction of increased awareness associated with declining drug use

 
"Behavior change is guided by the Theory of Reasoned Action (TRA: Ajzen & Fishbein, 1973, 1977) and draws also from social persuasion (McGuire, 1961, 1966, 1968) and communication theories (Hovland, Janis, & Kelley, 1953). According to the TRA, the influence of attitudes (i.e., subjective evaluations of behavior consequences) and beliefs (subjective norms and behavioral outcomes or expectancies) on behavior is mediated through intentions (i.e., future intent to engage the behavior). In other words, youth form impressions of whether drugs are good or bad, and they combine this information with normative beliefs (whether their close friends approve of drug use) and behavioral expectations (perceived benefits and negative consequences of drug use) toward drug use. These steps are necessary but not sufficient conditions, as the final decision to use drugs is guided by their behavioral willingness or intentions.", page 242

"To date, analyses of the media campaign efficacy have used traditional linear regression or correlation techniques to examine campaign effects. While this tactic has been useful to delineate the basic statistical associations between campaign awareness and drug use, a major weakness of this approach is that it fails to provide a developmental perspective and incorporate systematic features of change in either awareness or drug use.[..] Growth modelling is clearly a more definitive way to address the question of change and increasingly has been advocated as a means to assess prevention effects that unfold over time (Brown, Catalano, Fleming, Haggerty, & Abbott, 2005; Mason, Kosterman, Hawkins, Haggerty, & Spoth, 2003; Park et al., 2000; Taylor, Graham, Cumsille, & Hansen, 2000). [..] The age mixture within each round makes it imperative to estimate growth using age- cohort models", page 242-3


Slater 2011

CriterionScore/InfoNotes

In a well-conducted cohort study:  

The study addresses an appropriate and clearly focused questionWell covered"...(a) provide two simultaneous tests of autonomy and aspiration perceptions as mediators of impact on marijuana use as a consequence of exposure to each of these campaigns, b) conduct the first independent assessment of the ONDCP media campaign, which did not have a formal independent evaluation in place during the years of this study, and c) assess the simultaneous impact of a national campaign and a similar community/in-school effort.", page 12-13

Selection of subjects  

The 2 groups being studied are selected from source populations that are comparable in all respects other than the factor under investigationNot reported"3,236 students participated in at least one survey, with 48% males, 52% females and a mean age at baseline of 12.4 years (SD = 0.6); 75% were European-American, 11.5% African-American, and 13.5% of other racial backgrounds. One-quarter of the youth were of Hispanic ethnicity.", page 15

The study indicates how many of the people asked to take part did so, in each of the groups being studiedPoorly addressedOnly average: "The average rate of student participation in  each school was 32% of total student enrolment, lower  than the prior study because of stricter IRB requirements  being imposed on recruitment procedures. 57.1% of respondents provided data at all four measurement occasions; 27.2% provided data on three, 9.4% provided data on  two and 5.3% provided data on just one of the measurement  occasions. Missed surveys appear to be a matter more of  absenteeism or slips in getting students to survey sessions, than of panel mortality; 84.5% of participants filled out the  wave 1 survey, 86.2% wave 2, 86.1% wave 3, and 81.3%  wave 4.", page 15

The likelihood that some eligible subjects might have the outcome at the time of enrolment is assessed and taken into account in the analysisWell covered"Lifetime use of marijuana was measured at each measurement wave [..]", page 15

What percentage of individuals or clusters recruited into each arm of the study dropped out before the study was completed42.9%"The average rate of student participation in each school was 32% of total student enrolment, lower than the prior study because of stricter IRB requirements being imposed on recruitment procedures. 57.1% of respondents provided data at all four measurement occasions; 27.2% provided data on three, 9.4% provided data on two and 5.3% provided data on just one of the measurement occasions. Missed surveys appear to be a matter more of absenteeism or slips in getting students to survey sessions, than of panel mortality; 84.5% of participants filled out the wave 1 survey, 86.2% wave 2, 86.1% wave 3, and 81.3% wave 4.", page 15

Comparison is made between full participants and those lost to follow-up, by exposure statusNot reported 

Assessment  

The outcomes are clearly definedWell covered"Autonomy and Aspirations Inconsistent With Marijuana Use Autonomy inconsistent with marijuana use was measured using responses to four items following the phrase "Not using marijuana": 1) is a way to be true to myself; 2) is an important part of who I am; 3) is a way of being in control of my life; and 4) is a way of showing my own independence, where responses ranged from 1 = definitely disagree to 4 = definitely agree. Similarly, aspirations inconsistent with marijuana use were measured using the responses to three items following the phrase "Using marijuana would: 1) keep me from doing the things I want to; 2) mess up my plans for when I am older; and 3) get in the way of what is important to me." Because responses to each scale's items were heavily skewed, with 82% of respondents selecting "definitely agree" for all aspiration items and 84% of respondents selecting "definitely agree" for all autonomy items, each scale was dichotomized such that a "1" was assigned if all responses to the scale items were "definitely agree" and a "0" otherwise. The Cronbach's alpha values (Cronbach 1951) for each dichotomized measure were .9 or greater at each of the four waves.

Marijuana Use Lifetime use of marijuana was measured at each measurement wave using four questions: "How old were you the first time you used marijuana?", "How often in the last month have you used marijuana?", "How often in the last 3 months have you used marijuana?", and "Have you ever tried marijuana? (pot, grass, hash, etc.)?" If a subject responded affirmatively to any one question (or indicated an age when they first used marijuana), lifetime marijuana use was scored a "1", while an indication of never using marijuana resulted in a score of "0". The reliability for the scale was above 0.7 for the first two measurement occasions, .64 on the third occasion, and .69 at the fourth occasion.", page 15

The assessment of outcome is made blind to exposure statusNot applicableBlinding to exposure status was not applicable for this study

Where blinding was not possible, there is some recognition that knowledge of exposure status could have influenced the assessment of outcomeNot reported 

The measure of assessment of exposure is reliableWell coveredp. 15

Evidence from other sources is used to demonstrate that the method of outcome assessment is valid and reliableWell covered= 1.7

Exposure level or prognostic factor is assessed more than onceWell covered4 waves, page 17

Confounding  

The main potential confounders are identified and taken into account in the design and analysisAdequately addressesp. 16

Statistical analysis  

Have confidence intervals been provided?Well coveredStandard errors, e.g. Table 1 and 2, p. 18

Overall assessment of the study  

How well was the study done to minimise the risk of bias or confounding, and to establish a causal relationship between exposure and effect?

Code ++, +, or −
+ 

Taking into account clinical considerations, your evaluation of the methodology used, and the statistical power of the study, are you certain that the overall effect is due to the exposure being investigated?Fairly: selectivity (do no know if representative); no propensity scoring for national media campaign 

Are the results of this study directly applicable to the patient group targeted in this guideline?UnclearResults are applicable to US youth; it is unclear whether they are generalisable outside the US

Description of the study  

Do we know who the study was funded by?Public Funds (NIDA)"This research was supported by grant DA12360 from the National  Institute on Drug Abuse (NIDA) to the first author.", page 12

How many centres are patients recruited from?20 communities 

From which countries are patients selected? (Select all those involved. Note additional countries after 'Other')USA 

What is the social setting (i.e. type of environment in which they live) of patients in the study?Mixedp. 14

What criteria are used to decide who should be INCLUDED in the study?IRB requirements"The average rate of student participation in each school was 32% of total student enrolment, lower than the prior study because of stricter IRB requirements being imposed on recruitment procedures", page 15

What criteria are used to decide who should be EXCLUDED from the study?Exaggerators"Students who responded that they had tried all drugs listed including one that had been invented were considered exaggerators and were excluded from analyses; there were no more than 0.4% of such exaggerators in any given wave of data collection.", page 15

What intervention or risk factor is investigated in the study? (Include dosage where appropriate)(a) school- and community-based media intervention 'Be Under Your Influence' and (b) national anti-drug media campaign 'Above the Influence'p. 12

What comparisons are made in the study (i.e. what alternative treatments are used to compare the intervention/exposure with). Include dosage where appropriateExposure versus drug use/aspirations/autonomy; exposure x time versus drug  use/aspirations/autonomy 

What methods were used to randomise patients, blind patients or investigators, and to conceal the randomisation process from investigators?Randomisation: not applicable for mass media campaign, but done for 'Be Under Your Own Influence' school- and community-based media intervention 

How long did the active phase of the study last?Autumn 2005 to Spring 2009 (˜42 months) 

How long were patients followed up for, during and after the study?24 months 

List the key characteristics of the patient population. Note if there are any significant differences between different arms of the trial48% males, 52% females and a mean age at baseline of 12.4 years (SD = 0.6); 75% were European-American, 11.5% African-American, and 13.5% of other racial backgrounds One-quarter of the youth were of Hispanic ethnicityp. 15

Record the basic data for each arm of the study. If there are more than 4 arms, note data for subsequent arms at the bottom of the page Table 1 and 2, page 18

Record the basic data for each IMPORTANT outcome in the study. If there are more than 4, note data for additional outcomes at the bottom of the page Table 1 and 2, page 18

Notes. Summarise the authors' conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your questionResults indicate that earlier effects of the 'Be Under Your Own Influence' intervention replicated only in part and that the most plausible explanation of the weaker effects is high exposure to the similar but more extensive ONDCP 'Above the Influence' national campaign. Self reported exposure to the ONDCP campaign predicted reduced marijuana use, and analyses partially support indirect effects of the 2 campaigns via aspirations and autonomy 

 SD: standard deviation
IRB= Institutional Review Board, is a committee that has been formally designated to approve, monitor, and review biomedical and behavioral research involving humans
 
Table 3. Measurement scales used in included studies

StudyWas a specific scale developed? (Yes/no/unclear)Measurement scale(s) used ReferenceWas the scale adapted?
(Yes/no/unclear)

Palmgreen 1991NoSensation seeking Scale, Form VZuckerman, M (1979). Sensation seeking: beyond the optimal level of arousal. Hillsdale, NJ:Lawrence Erlbaum Associates, Inc.No

NoInstruments used in a continuing survey of young people by the Institute for Social Research at the University of Michigan
(NB to measure levels of use of illicit drugs) 
Johnston LD, Bachman JG, O'Malley PM (1982). Monitoring the future: questionnaire responses from the nations' high school seniors, 1981. Ann Arbor: University of Michigan, Survey Response Centre, Institute for Social ResearchYes

YesBehavioural Intention Index
p. 221 "immediately after the second viewing of the PSA, subjects were asked "If you wanted information about alternatives to drug use, how likely it is, on a scale of 1 to 5, that you would call na 800 hotline?"
n/an/a

YesAttitude towards drug use
p. 222 "After behavioural intention was measured, subjects were asked to indicate on a scale of 1 to 5 how they felt about their personal use of drugs in relation to each of six adjectives word pairs."
n/an/a

Kelly 1992UnclearNot mentionedn/an/a

Polansky 1999NoDrug Attitude Scale
(12 items on a Likert scale)
Swisher JD, Horan JJ (1973). The Pennsylvania State University Evaluation Scales. In LA Abrams, E Garfield & JD Swisher (eds). Accountability in drug education: a model for evaluation (pp 87-99). Washington, DC: Drug Abuse CouncilUnclear
(: in the text is mentioned "updated version" but no further clarification)

NoTentative Drug Use Scale
(10 items scale)
Horan JJ, Williams JM (1975). The tentative drug use scale: a quick and relatively problem free outcome measure for drug abuse prevention projects. Journal of Drug education; 5: 381-4No

YesHelp-Seeking Questionnaire and Knowledge Questionnaire
p. 190 "two 10-items achievement rests were developed for this study"
n/an/a

NoDrug Conformity Scale
(16 questions reflecting varying levels of assertive competency)
Horan JJ, Williams JM (1982). Longitudinal study of assertion training as a drug abuse prevention strategy. American Educational Research Journal; 19: 341-51No

Palmgreen 2001 (Stephenson 1999)YesBeliefs
12 marijuana-related beliefs about occasional use of marijuana and 12 belief items about regular marijuana use were assessed on a 4-point scale with the response options of disagree strongly, disagree somewhat, agree somewhat and agree strongly
n/an/a

YesAttitudes
Seven marijuana-related attitudes about occasional use and 7 items about regular use were assessed on a 4-point scale, with the response options of disagree strongly, disagree somewhat, agree somewhat and agree strongly
n/an/a

YesIntentions
Participants were asked their intent to engage in experimental or regular marijuana use in the future. With 2 items on a 3-point scale with the response options  probably will not, probably will and definitely will
n/an/a

Miller 2000YesUse
Recent drug use was measured by asking respondents about the  frequency and recency with which they used 10 types of drugs (using  commonly recognised names): cannabis, cocaine, other stimulants, tranquillisers, sedative-hypnotics, hallucinogens, opioids, phencyclidine, amyl and butyl nitrates and inhalants such as glue, paint or gasoline (4- point scale ranging from 1 = never to 4 = at least once in the past month)
n/an/a

YesRisks perception
Risk assessment included 13 items regarding the extent to which students perceived risk or consequences related to alcohol or other drug use. Personal risk for alcohol and other drug problems was judged relative to students' perceptions of "most people" (ranging from 1 = higher than most people to 3 = lower than most people)
n/an/a

Palmgreen 2001NoBrief Sensation Seeking ScaleHoyle RH, Stephenson MT. The sensation seeking scale for adolescents. In: Lennox RD, Scott-Lennox JA, Cutler BL, eds. Applied Psychometrics for Health Outcomes Research. Chapel Hill, NC: Health Statistics Lab. No

Fishbein 2002YesSpecifically developed instrument
p. 241 "the instrument for the study consisted of  questionnaire with 3 parts."

First: demographic questions
Second: series of questions on realism/content/recall of intervention
Third: assessment of the respondent perceptions of the danger and harmful effects of engaging in 8 risky behaviours
- perceived danger = 1 item per behaviour on yes/no basis
- perceived harmfulness = 1 item per behaviour on a 5-point scale
- perceived norms = 1 item per behaviour on a 5-point scale
n/an/a

Yzer 2003YesA specific questionnaire was developed for the study. Available upon request by the authors (p. 135)

Intention to use marijuana: 1 to 2 (depending on the first answer) items using a 4-point scale
Attitude: 4 items using a 7-point scale
Outcome beliefs: 36 items using a 5-point scale
n/an/a

Slater 2006NoSelected items from the American Drug and Alcohol Survey
Alcohol lifetime score: 3 items
Smoking lifetime score: 3 items
Marijuana lifetime score: 5 items
American Drug and Alcohol Survey, with permission by the Rocky Mountain Behavioural Science InstituteNo

Zhao 2006YesIntentions not to use/to reduce use/to stop usen/aNo

UnclearAttitudes towards illicit drug use: 7-point scale from -3 (bad/foolish/...) to +3 (good/wise/...)n/an/a

YesPerceptions (including perceptions of peer norms and perceptions about illicit drug use): 5-point scales from -2 to +2. "Although we did some analyses at the level of individual beliefs, we generally used two types of belief clusters in our analyses"n/aNo

Czyzewska 2007UnclearDeclared intention to use marijuanan/a

UnclearAttitudes towards illicit drug use (pre-test explicit attitudes): 10-point Likert scalesn/a

YesAttitudes towards illicit drug use (post-test implicit attitudes): IAT test. "Two computerized Implicit Association Tests (IAT) were designed to assess implicit attitudes to tobacco and marijuana. [..] The only difference to the standard IAT procedure was the extended number of practice trials to 40 in order to reduce the typical effect of order in which the combined categorization tasks are performed"See Table 1 for IAT test content

Scale was adapted from: Greenwald AG, McGhee DE, Schwartz JLK (1998). Measuring individual differences in social cognition: The Implicit Association Test. Journal of Personality and Social Psychology; 74: 1464-80

With updates from: Greenwald AG, Nosek BA, Banaij MR (2003). Understanding and using the Implicit Association Test: An improved scoring algorithm. Journal of Personality and Social Psychology; 85(2):197-216
Yes

UnclearAttitudes towards illicit drug use (post-test explicit attitudes): 3 sets of 7 5-point scales (= 21 5-point scales)n/a

Hornik 2006NoNational Survey of Parents and Youth (NSPY).
3 types of measures: recalled exposure to anti-drug messages aired by the campaign and other sources; cognitions and behavior related to marijuana, as outcomes; and individual and household characteristics, including a wide range of variables known to be related to drug cognitions and use and to exposure to anti-drug messages
http://archives.drugabuse.gov/initiatives/westat/No

Scheier 2010 YesAlcohol and Drug Use
Assessment of alcohol and drug use relied on an Anonymous Computer Assisted Self-report Interview (ACASI)
n/an/a

Fang 2010NoOccasions of use in the past 30 days ± standard error, SENone, but it is a standard question in this fieldNo

UnclearIntentions not to use/to reduce use/to stop use: 5-point scales; higher scores are better. No additional information Unclear

Lee 2010No90-day marijuana use: "items were adapted from the Global Appraisal of Individual Needs-I"Dennis ML, Titus JC, Diamond G, Donaldson J, Godley SH, Tims FM. The CYT Steering Committee (2002). The cannabis youth treatment (CYT) experiment: Rationale, study design and analysis plans. Addiction; 97(Suppl 1): 16–34Yes

NoIntentions not to use/to reduce use/to stop use: 4-point score (higher = more "contemplation"). "Contemplation to change marijuana use was assessed with four items (alpha = 0.79) adapted from the Readiness to Change Questionnaire (RTCQ)"Heather N, Gold R, Rollnick S (1991). Readiness to change questionnaire: User's manual. (Tech. Rep. 15). Kensington, Australia: National Drug and Alcohol Research Center, University of New South WalesYes

NoKnowledge about the effects of illicit drugs on health: negative consequences due to marijuana use. 5-point score (from 0 = never to 4 = more than 10 times). "Consequences of marijuana use were assessed using the Rutgers Merijuana Problem Index (RMPI)"White HR, Labouvie EW, Papadaratsakis V (2005). Changes in substance use during the transition to adulthood: A comparison of college students and their non-college age peers. Journal of Drug Issues; 35: 281–306Unclear

Newton 2010NoFrequency of cannabis use: times per week ± SE in the past 12 months "Cannabis use was assessed from a questionnaire in the 2007 National Drug Strategy Household Survey (NDSHS) that identified the frequency of use of cannabis [1]."Australian Institute of Health and Welfare. 2007 National Drug Strategy Household Survey: First Results. Canberra: AIHW; 2008Yes

NoAttitudes towards illicit drug use: score ± SE "Attitudes towards cannabis were measured by four items from the Life Skills Training Questionnaire [37], which has acceptable internal consistency (a = 0.86)."National Health Promotion Associates (NHPA) Incorporated. Life Skills Training Questionnaire—Middle School. New York: NHPA; 2004Unclear

NoKnowledge about the effects of illicit drugs on health: score ± SE
"The cannabis knowledge questionnaire was adapted from the Cannabis Quiz and included 16 items [33]."
Bleeker A, Malcolm A. The Cannabis Quiz. Sydney: Manly Drug Education and Counselling Centre; 2001Yes

NoKnowledge about the effects of illicit drugs on health: score ± SE
"Cannabis harms were assessed with six questions derived from the Adolescent Cannabis Problems Questionnaire (test–retest reliability, r = 0.91) [35]."
Martin G, Copeland J, Gilmour S, Gates P, Swift W. The adolescent cannabis problems questionnaire (CPQ-A): psychometric properties. Addictive Behaviors 2006; 31: 2238–48No

Schwinn 2010NoPast 30-day drug use (marijuana): occasions of use (0 to 40)
"...adapted from the CDC's Youth Risk Behavior Survey (YRBS; Centers for Disease Control and Prevention 2005), asked girls to report how many times in the past month and week they used alcohol, cigarettes, marijuana, cocaine, inhalants, methamphetamines, and ecstasy. Response options ranged from "0 times" to "40 or more times." Test–retest reliability for YRBS items is 0.82 to 0.95 (Centers for Disease Control and Prevention 2004)"
Centers for Disease Control and Prevention. (2005). Youth Risk Behavior Survey. Retrieved February 20, 2009, from http://www.cdc.gov/healthyyouth/yrbs/Yes

NoPast 30-day drug use, poly drug use (cigarettes, marijuana cocaine, inhalants, met., ectasy): 7-point score (0 to 6). Same as aboveSame as aboveYes

NoPast 30-day drug use, total substance (= poly drug use + alcohol): 8-point score (0 to 7). Same as aboveSame as aboveYes

Idaho Meth 2010;

Colorado Meth 2011;
Georgia Meth 2011;
Hawaii Meth 2011;
Wyoming Meth 2011
NoPast-year and past-month use of methamphetamine: "Have used meth in past year"; "Have used meth in past month"n/aUnclear

UnclearAttitudes towards illicit drug use: "Please indicate how much you approve or disapprove of the following activities." (Strongly disapprove, strongly/somewhat approve)n/aUnclear

UnclearPerceptions (including perceptions of peer norms and perceptions about illicit drug use: binary and categorical questions, such as "How difficult, or easy, do you think it would be for you to get each of the following types of drugs?" (easy, difficult) and "Please indicate how much risk, if any, you think there is involved in each of the following activities." (Great risk, great/moderate risk, little/no risk)n/aUnclear

Slater 2011YesDescription of study measures and survey components (p. 15)

Autonomy inconsistent with marijuana use: 4 items on a 4-point scale
Aspirations inconsistent with marijuana use: 3 items on a 4-point scale
Lifetime marijuana use: 4 items
Exposure to ONDCP's campaign: 1 item on a 3-point scale
n/an/a

Carpenter 2011NoLifetime marijuana usen/aUnclear

NoPast-month marijuana usen/aUnclear

NoAlcohol usen/aUnclear

 IAT: Implicit Association Test
n/a: not applicable
ONDCP: Office of National Drug Control Policy
SE: standard error