The number of users for the ‘not-treated’ intervention subgroups is estimated by applying the proportion of users in the control group to the number of individuals in the intervention groups that did not attend an intervention (‘not-treated’ subgroup users = control group user proportion × the number of intervention group individuals not actually attending). The number of users in the ‘treated’ intervention subgroups is then computed as the observed number of users in the entire intervention groups minus the estimated number of users among the ‘not-treated’ subgroups, with negative results rounded to 0. ISFP: Iowa Strengthening Families Program; PDFY: Preparing for Drug Free Years.
[Commentary] FINDING A PATH TO MORE REASONABLE CONCLUSIONS ABOUT PREVENTION: A RESPONSE TO MIDFORD
Article first published online: 28 JUN 2008
© 2008 The Authors. Journal compilation © 2008 Society for the Study of Addiction
Volume 103, Issue 7, pages 1171–1173, July 2008
How to Cite
SPOTH, R., TRUDEAU, L., REDMOND, C. and SHIN, C. (2008), [Commentary] FINDING A PATH TO MORE REASONABLE CONCLUSIONS ABOUT PREVENTION: A RESPONSE TO MIDFORD. Addiction, 103: 1171–1173. doi: 10.1111/j.1360-0443.2008.02270.x
- Issue published online: 28 JUN 2008
- Article first published online: 28 JUN 2008
In his commentary ‘Is this the path to effective prevention?’, Dr Midford expresses his ‘unease about the paradigm that underpins effort’ in the area of universal preventive intervention research. He raises general issues about the role of universal preventive interventions vis-à-vis other strategies that potentially mitigate the consequences of substance misuse, as well as issues about methods for evaluating intervention efficacy. He supports his critical view of the research on universal prevention with a critique of our paper , taking issue with one-tailed significance tests and presenting ‘re-analyses’ of our data. We commend critical reviews of research on preventive interventions that highlight important issues to consider; we take exception, however, to what we view as a misleading articulation of issues regarding our paper and the questionable reasoning used in conducting re-analyses of our data. In addition, treatment of more general issues fails to adequately consider the range of relevant literature.
A major issue concerns Midford's assertion that our use of an intent-to-treat analysis magnifies effects by inflating sample size. Intent-to-treat analyses are considered widely to be the gold standard for initial testing of intervention efficacy, avoiding intervention self-selection biases and other potential confounds that may occur in subgroup analyses . Moreover, intent-to-treat analyses are assumed generally to be more conservative, often deflating observed intervention effects by including data from individuals who did not receive an intervention in the analyzed intervention group.
Midford's ‘as-treated’ re-analyses do not incorporate information on who did or did not actually receive an intervention, instead estimating the number of attendees from intervention participation tables. As such, his analyses cannot be described legitimately as ‘as-treated’. Also, he assumes that the proportion of substance users in his reduced sample would be the same as that observed in our full sample. This assumption is not defensible because the justification for conducting ‘as-treated’ analyses assumes that only actual participants in the intervention would show intervention benefits. To be consistent with this justification, the proportion of users in the ‘not-treated’ subgroups in the intervention conditions in study 1 (not presented in Midford's tables) would be comparable to the proportion of users as the control group; for study 2, LST (Life Skills Training)-only rates of drug misuse should be applied to the portion of the SFP (Strengthening Families Program): 10–14 + LST group who did not attend SFP: 10–14 (Midford ignores the fact that SFP: 10–14 non-attendees actually received LST). Although we maintain that intent-to-treat analyses are the most appropriate in our case, we demonstrate how a more appropriately informed approximation of ‘as-treated’ analyses shows intervention results considerably stronger than reported by Midford (see Tables 1 and 2).
|12th grade past year misuse of prescription narcotics||Young adult life-time misuse of prescription narcotics||Young adult life-time misuse of prescription barbiturates|
|Yes (%)||No (%)||Total (%)||P-value two-tailed Fisher's exact test||Yes (%)||No (%)||Total (%)||P-value two-tailed Fisher's exact test||Yes (%)||No (%)||Total (%)||P-value two-tailed Fisher's exact test|
|ISFP (not treated)||1.30||34||4.61||53||1.64||53|
|PDFY (not treated)||0.96||25||2.45||28||0.87||28|
|11th grade life-time prescription drug misuse||12th grade life-time prescription drug misuse|
|Yes (%)||No (%)||Total (%)||P-value two-tailed Fisher's exact test||Yes (%)||No (%)||Total (%)||P-value two-tailed Fisher's exact test|
|LST + SFP: 10–14||5||132||137||0.1202||10||127||137||0.3090|
|LST + SFP: 10–14 (not treated)||24.8||379||31.2||379|
|LST + SFP: 10–14 (treated)*||0||137||137||0.0001||3||134||137||0.0015|
Independent of methodological problems with Midford's re-analyses, it is important to note that such analyses are also misguided on theoretical grounds. They ignore published research discussing mechanisms of long-term intervention effects among non-participants in a community, occurring via peer and/or parent social networks . Based on this work, reductions in substance use among those directly receiving interventions could be expected, over time, to produce reductions in substance use among non-participating peers, further justifying analyses incorporating the full sample.
A more minor issue concerns our use of one-tailed significance tests. Negative intervention effects may be conceived as a possibility; that conception, however, would be unreasonable because of the preponderance of evidence for the specific interventions examined. Results in numerous refereed journal reports (, http://www.ppsi.iastate.edu) clearly justify directional hypotheses and, therefore, one-tailed significance tests. In any event, our use of directional hypotheses is transparent; any reader who disagrees with this usage could readily convert the reported P-values accordingly. Also, it is simply inaccurate to state that an alpha was ‘set at 0.10’ (p. 1169 ). Our reports follow the social science standard of reporting P values associated with a range of alpha levels, clearly labeling findings at or below 0.10 as ‘marginally significant’ (e.g. p. 1166 ).
We believe that the critical reader would want to assess two central but distinct questions about our paper: ‘What are the most valid and reasonable conclusions to be drawn about intervention efficacy from study observations?’ and ‘What is the practical significance of those findings?’. Concerning the first question, the paper describes studies that are designed to address threats to internal, construct, and statistical conclusion validity, along with study limitations. Pertaining to practical significance, our paper presents relevant interpretive guidelines. Overall, readers are in a good position to draw their own conclusions.
Concerning the more general issues noted in the opening paragraph, Midford draws upon a subset of critical papers, failing to represent the full range of reviews of the evidence base for preventive interventions (the lead author recently found more than 24 reviews of relevance (e.g. [6–8]). Midford fails to address adequately the rationale for universal interventions, and his conclusions ignore relevant literature describing the most appropriate application of universal interventions, in conjunction with selective and indicated ones . We support strongly an open dialogue among scientists with varying positions upon the important issues raised by Midford, but believe they should be guided by full consideration of the relevant literatures and follow accepted standards for preventive intervention science (see http://www.preventionscience.org).
- 3Fundamentals of Clinical Trials, 3rd edn. New York: Springer; 1998., ,
- 5Blueprints for Violence Prevention: Life Skills Training (Book Five). Boulder: University of Colorado, Institute of Behavioral Science, Center for the Study of Prevention and Violence; 1998. Available at: http://www.colorado.edu/cspv/blueprints
- 6Family-based approaches to substance abuse prevention. J Prim Prev 2002; 23: 49–114.,