An extensive methodological literature suggests that measurement error severely undermines survey research on substance use. In many contexts, survey reports of drug use may have limited reliability and validity. Limitations in data reliability are supported by studies investigating life-time drug use reports over multiple waves of interviews. Researchers have identified the phenomenon of ‘recanting’, or the denial of previously asserted life-time substance use in a follow-up or re-inteview. This phenomenon has been identified in both school-based (Fendrich & Rosenbaum 2003) and household (Fendrich & Vaughn 1994) surveys. With respect to validity, recent studies supplementing epidemiological surveys with individually administered drug tests (e.g. Fendrich et al. 2004), provide strong evidence that drug use is under-reported by adults.
Survey research is the core methodology of epidemiologists as well as of clinical and prevention researchers. For school-based researchers it would be impractical to replace survey research with methods less susceptible to response bias, such as biological testing. Therefore, studies that bring us closer to understanding the extent, sources and correlates of survey measurement error, such as the paper by Percy et al. (2005) in the current issue of Addiction, are of critical importance. The Percy et al. paper is characterized by several important innovations. To our knowledge, it is the first published paper to tackle the recanting issue by focusing on a non-US school-based sample. It is also noteworthy for its focus on the reporting behavior of relatively young adolescents (average age of 12.5 years). By going beyond the descriptive and systematically evaluating potential causes of recanting, the work holds out the prospect of guiding improvements in future school-based drug abuse research studies.
Percy et al. (2005) suggest that recanting of previously disclosed drug use in a 1-year follow-up was highly prevalent among adolescents in Northern Ireland. Recanting was more pronounced for more ‘serious’ substances such as heroin and cocaine than it was for other substances such as alcohol, marijuana and tobacco. For alcohol and cannabis, initially reported use frequency was inversely associated with recanting.
The finding, unique to this study, that those students who reported exposure to antidrug education were more likely to recant is particularly alarming in light of the fact that school-based drug education/prevention programs rely on survey data to monitor changes in actual behavior (not changes in reporting behavior) to evaluate program impact. The finding that recanting of life-time substance use, especially serious drug use, is a highly prevalent phenomenon in longitudinal school samples, confirms data independently obtained from a US school-based sample (Fendrich & Rosenbaum 2003). Substance use recanting is an undeniable fact that has been observed repeatedly across multiple study contexts in both Europe and the United States. Recanting is a cross-culturally generalizable construct.
While we can identify correlates of recanting, its immediate ‘cause’ is difficult to disentangle. Recanting may reflect denial of previously disclosed ‘actual’ behavior, exaggeration about behaviors that never occurred, or some combination of both types of behavior. Indeed, the appropriate explanation may vary depending on the sample being assessed (or even within the same sample). Given that drug use is often considered to be ‘sensitive’ behavior, some have assumed that in samples of adults (Fendrich & Kim 2001), recanting mainly indicates a reluctance to repeatedly disclose actual behavior on surveys due to increasing respondent discomfort. Or, perhaps drawing on the literature evaluating psychological processes leading to survey responses (Tourangeau, Rips & Rasinski 2000), the errors encountered are less the result of intentional distortions (or ‘response editing’), but rather the result of poor comprehension, forgetting, or even carelessness (Fendrich & Mackesy-Amiti 2000). Taking into account the young age of their sample and the sizeable recanting rate differences across substances, the weight of the evidence from the Percy et al. (2005) study may suggest that initial exaggeration may account for much of the recanting in this particular sample. The clustering of recanting within certain schools in this study suggests that there may also be certain contextual and administrative issues that factor into the problem.
Appropriate design strategies for preventing recanting partly depend on its actual cause. As we have noted previously (Fendrich & Rosenbaum 2003), if intentional exaggeration is the cause, question formats in the initial interview that request specific details about the use experience, such as timing of first and last use, the number of use occasions and the amount of substance used, may lead to more accurate initial reporting. If denial on re-interview is the cause of recanting, procedures that minimize concerns about disclosure, such as added assurances of confidentiality and private, automated administration procedures (audio computer assisted self-interview; ACASI) may lead to more consistent reporting at follow-up. In either case, researchers might consider using ACASI-based probes at follow-up to provide each subject with an opportunity to resolve inconsistent responses. While the use of ACASI may considerably raise the cost of data collection (possibly precluding the seemingly efficient group administration format), the potential benefit of greatly improved data quality make this an attractive alternative for researchers to consider.
In order to eliminate the problem of recanting, researchers might also consider eliminating questions about life-time drug use from their surveys. Although life-time use questions are conventionally asked in many drug use surveys, their utility is questionable, especially for young school-aged samples where survey administration periods are likely to be temporally close to initiation age and current use behavior. Researchers may consider limiting their assessment battery to more specific questions about recent drug use in more clearly defined time-frames (past year, past 30 days, past week). Unfortunately, while this strategy may resolve the recanting problem, it may not address possible validity problems associated with responses to other drug use questions. Even though recanting may indicate flaws associated with life-time use measures, this problem may be symptomatic of more pervasive validity threats that affect all substance use questions, irrespective of time frame.
All this underscores the importance of solid methodological research on substance abuse (Midanik & Greenfield 2004). As was noted previously (Fendrich & Johnson 2001), the need for methodological research focused specifically on examining the validity of school-based substance use surveys is particularly acute. All the research on recanting, including the work by Percy et al., has been post hoc, based on secondary analysis of surveys implemented for studies not designed specifically to address reliability and validity issues. We need carefully designed longitudinal studies that incorporate in-depth interviews that present subjects with information about their inconsistent reporting behavior. Subjects need to be asked to explain their answers and to discuss their motivations for responding. Human subjects concerns arising from such studies will prove challenging, especially for US researchers. The potential societal benefits of such studies clearly outweigh the potential risks. In the absence of comprehensive, pre-planned research incorporating both qualitative and quantitative data-gathering techniques, we will never understand the cause of recanting with sufficient certainty to prescribe a remedy.