SEARCH

SEARCH BY CITATION

Keywords:

  • brief alcohol interventions;
  • control group drinking reductions;
  • regression to the mean

Abstract

  1. Top of page
  2. Abstract
  3. Introduction and Aims
  4. Reductions in control group alcohol consumption over time
  5. Potential sources of control group reductions: the role of bias related to research design
  6. Studies designed specifically to evaluate assessment reactivity
  7. Strategies to limit inadvertent moderators of control group drinking
  8. Conclusions
  9. References

Issues. Reductions in control group consumption over time that are possibly related to research design affect the impact of brief alcohol interventions (BAI) in clinical settings. Approach. We conducted a systematic review to identify research design factors that may contribute to control group change, strategies to limit these effects and implications for researchers. Studies with control group n > 30 were selected if they published baseline and outcome consumption data, conducted trials in clinical settings in Anglophone countries and did not censor gender or age. Key Findings. Among 38 studies cited in 20 reviews through October 2009, 16 met criteria (n = 31–370). In 54%, controls received alcohol specific handouts, advice and/or referral. Both the number and depth of assessments were highly variable. The percentage change in consumption ranged from0.10 to0.84 (mean0.32), and effect size from 0.04 to 0.70 (mean 0.37). Published data were insufficient for meta-analysis. Implications. Researchers should consider strategies to reduce the impact of research design factors, such as procedures to enhance sample diversity, blind subjects to study purpose to limit social desirability bias, reduce the number and depth of instruments (assessment reactivity), and finally, analytic techniques to decrease the impact of outliers and regression to the mean. Conclusions. This review identifies problems with retrospective analysis of predictors of control group change, and underscores the need to design prospective studies to permit identification, quantification and adjustment for potential sources of bias in BAI trials.[Bernstein JA, Bernstein E, Heeren TC. Mechanisms of change in control group drinking in clinical trials of brief alcohol intervention: Implications for bias toward the null. Drug Alcohol Rev 2010;29;498–507]


Introduction and Aims

  1. Top of page
  2. Abstract
  3. Introduction and Aims
  4. Reductions in control group alcohol consumption over time
  5. Potential sources of control group reductions: the role of bias related to research design
  6. Studies designed specifically to evaluate assessment reactivity
  7. Strategies to limit inadvertent moderators of control group drinking
  8. Conclusions
  9. References

Two decades ago teams of clinicians, psychologists and alcohol researchers began systematic testing of the notion that brief alcohol intervention (BAI) in the clinical setting with problem drinkers could decrease alcohol consumption and improve health status. Twenty meta-analyses later, short-term reductions in consumption and clinically meaningful health benefits have been fully documented in health-care settings, principally in primary care [1–20]. However, open questions remain about long-term effects, mechanisms of change, best practices, and applicability of results across diverse settings and sub-populations. The field has come of age, ‘middle age’ according to Babor and colleagues [21], and with maturity comes the responsibility to examine the quality of the scientific evidence and explore remaining methodologic issues.

Perhaps the most puzzling of these emerging methodologic concerns is the degree to which control groups in studies of BAI reduce alcohol consumption over time. Effect sizes for clinical BAI are small, typically a mean difference of four drinks less per week (38 g), a decrease of 12% [13]. It has become almost a convention for published reports to suggest that the impact of BAI and treatment may be larger than researchers areable to demonstrate because of bias introduced by assessment and other necessary elements of research design [2,21–29]. The topic is particularly timely in light of recent publication of several negative BAI trials. This paper reviews design factors that may contribute to control group change, strategies that are currently being tested to limit bias and the implications of unavoidable sources of bias for interpretation of intervention results.

Reductions in control group alcohol consumption over time

  1. Top of page
  2. Abstract
  3. Introduction and Aims
  4. Reductions in control group alcohol consumption over time
  5. Potential sources of control group reductions: the role of bias related to research design
  6. Studies designed specifically to evaluate assessment reactivity
  7. Strategies to limit inadvertent moderators of control group drinking
  8. Conclusions
  9. References

In a meta-analysis of BAI in a variety of settings, Jenkins et al.[30] present evidence for a clear pattern of control group reductions, with decreases in a wide range of consumption outcomes. Among 16 studies in English language countries that had data available for quantitative analysis, they found 15 with control group reductions from 11% to 46%.

Because this systematic review and meta-regression study restricted inclusion to primary studies covered by systematic reviews and meta-analyses published between 1995 and 2005 (11 review papers and 16 studies), we updated their list of studies to include all randomised, controlled Anglophone trials of BAI in clinical settings cited in reviews published from 2005 through 2009. In order to improve comparability of studies, we excluded those outside of the clinical setting, conducted in non-English speaking countries, or single gender- or youth-focused. We located a total of 20 reviews of BAI in Medline, CINAHL, PSYCHINFO, PUBMED and EMB Reviews [1–20]. These systematic reviews cited 38 randomised, controlled studies offering control group consumption data for BAI in the clinical setting in Anglophone countries. We rejected 22 of these 38 studies: 11 [31–41] for gender or age restrictions other than exclusion of minors, one [42] for small control sample at follow up and 10 [43–52] for absence of data needed to analyse control group changes in consumption. Table 1 describes the results of this search, and lists the design characteristics of the 16 studies [28,34,53–66] that met criteria for inclusion, the consumption outcome parameters measured and the percentage change in each study outcome from baseline to final assessment.

Table 1. Control group changes in consumption from baseline to follow up
Author, year # controls setting modalityBL inclusion criteriaControl group proceduresAssessment measures: #, timing (months)# and type of dimensions of measurementConsumption measuresControl group changes: decrease: BL (SD), F/U (SD)Effect size: % decrease Cohen's d
  1. *Insufficient data for Cohen's d. AUDIT, Alcohol Use Disorders Identification Test; BL, baseline; NIAAA, National Institute on Alcohol Abuse and Alcoholism.

Blow et al. 2006 [53] n = 120 ED patientsHigh-risk & dependent drinkers12-page booklet: alcohol use, consequences, safe limits3 BL 3, 65 Q/F screen drinking diary dependency screen consequences treatment historyAt 6 months: mean # drinks per week mean # binge episodes[DOWNWARDS ARROW]7.7: BL 21.3 (20.2), F/U 13.6 (16.8) [DOWNWARDS ARROW]2.8: BL 7.5 (6.6), F/U 4.7 (6.1)−0.36 d = 0.42 −0.37 d = 0.44
Crawford et al. 2004[54] n = 195 ED patientsHigh-risk & dependent drinkersGeneral information and advice3 BL, 3, 124 Q/F screen drinking diary psychosocial scales health surveyAt 12 months: mean units per drinking session[DOWNWARDS ARROW]5.8, BL 21·8 (14.0), F/U 16·0(15.6)−0.27 d = 0.39
Lock et al. 2006 [55] n = 42 primary careHigh-risk onlyRN Advice to ‘Think about drink’3 BL 6, 124 Q/F screen drinking diary consequences health surveyAt 12 months: mean units per week[DOWNWARDS ARROW]6.9, BL 26.5(29.8), F/U 19.6 (23.6)−0.26 d = 0.26
Fleming et al. 1999[34] n = 70 primary careHigh-risk & dependent drinkerGeneral health booklet Advice to f/u concerns with PCP4 BL, 2, 4, 124 drinking diary psychosocial scales consequences biomarkerAt 12 months: mean # drinks per 30 days mean # binge episodes per 30 days[DOWNWARDS ARROW]11, BL 35.5 (42.9), F/U 24.5(27.6) [DOWNWARDS ARROW]1.6 BL 3.0 (6.4), F/U 1.3(3.2)−0.31 d = 0.34 −0.53 d = 0.35
Kunz et al. 2004 [56] n = 104 ED patientsHigh-risk & dependent drinkersInformation and handout2 BL, 32 Q/F screen dependence screenAt 3 months: mean # drinks per week[DOWNWARDS ARROW]16.2, BL 36.3 (32.1) F/U 20.1(26.8)−0.45 d = 0.55
Curry et al. 2003 [57] n = 100 primary careHigh-risk onlyUsual care3 BL, 3, 125 Q/F screen drinking diary consequences readiness psychosocial scalesAt 12 months: mean # drinks per week[DOWNWARDS ARROW]4.1, BL 13.6, F/U 9.5−0.30*
Maisto et al. 2001 [58] n = 85 primary careHigh-risk and dependent drinkers (AUDIT ≥8)Usual care7 BL x 2, 1, 3, 6, 9, 12 months5 Q/F screen dependence screen drinking diary consequences readinessAt 12 months: mean # drinks per drinking day[DOWNWARDS ARROW]1.5, BL 6.0, F/U 4.5−0.25*
Ockene et al. 1999 [59] n = 233 primary care>NIAAA low-risk criteria; dependency not excluded, but only 2% of sampleDischarged with booklet and written advice on general healthn = 2 BL, 6n = 5 Q/F screen drinking diary dependency screen consequences treatment historyAt 6 months: mean drinks per week[DOWNWARDS ARROW]3.1, BL 16.4 (12.1), F/U 13.3 (12.7)−0.19 d = 0.25
Watson 1999 [60] n = 31 inpatient wardsHigh-risk and dependent drinkers (AUDIT range 1–35)Usual care2 BL, 12 months4 Q/F screen drinking diary consequences biomarkerAt 12 months: mean # units per week[DOWNWARDS ARROW]14.7, BL 45.2 (42.9),F/U 30.5 (30.0)−0.33 d = 0.40
Fleming et al. 1997[28] n = 370 primary careHigh-risk drinkers; excluded if history of withdrawal symptomsGeneral health booklet3 BL, 6, 125 Q/F screen drinking diary consequences psychosocial scales collateral reportAt 12 months: mean # drinks per week binge drinking episodes in 30 days[DOWNWARDS ARROW]3.4, BL 18.9 (11.8),F/U 15.5 (12.9) [DOWNWARDS ARROW]1.1, BL 5.3 (5.0), F/U 4.2 (5.5)−0.18 d = 0.28 −0.21 d = 0.21
Senft et al. 1997[61] n = 215 primary careHigh-risk onlyUsual care3 BL, 6, 124 Q/F questions Drinking diary Biomarker Health-care utilisation surveyAt 12 months: mean #drinks/[DOWNWARDS ARROW]6.2, BL 16.45, F/U 10.2−0.38*
Israel et al. 1996 [62] n = 35 primary careHigh-risk and dependent drinkers (>3 drinks per day or 8 days per month of binge episodes or CAGE+Feedback from assessment and advice2 BL, 12 months5 Q/F screen dependence screen injury questionnaire biomarker psychosocial scalesAt 12 months: drinks per 4 weeks[DOWNWARDS ARROW]61.7, BL 136.6 (94.7), F/U 74.9 (79.3)−0.45 d = 0.70
Richmond et al. 1995[63] n = 72 primary careHigh-risk & dependent drinkers (<350 g per week (males), 210 (females)Usual care3 BL, 6 12 months5 Q/F screen drinking diary consequences dependence screen\biomarkerAt 12 months: Q/F consumption in units[DOWNWARDS ARROW]BL 37.5 (19.9), F/U 33.8 (29.9)0.10 d = 0.04
Rowland and Maynard 1993[64] n = 126 inpatient wardsHigh-risk & dependent drinkersConsultant informed re high-risk drinking (advice not required)2 BL, 12 months5 Q/F screen dependence screen drinking diary consequences collateral reportAt 12 months: mean consumption units[DOWNWARDS ARROW]37, BL 44.3, F/U 7.3−0.84*
Wallace et al. 1988 [65] n = 322 primary careHigh-risk & dependent drinkersUsual care3 BL, 6, 123 Q/F screen dependence screen biomarkerAt 12 months: weekly alcohol units[DOWNWARDS ARROW]BL 63.7 (34.1), F/U 55.6 (32.3)−0.13 d = 0.24
Heather et al. 1987 [66] n = 32 primary careHigh-risk & dependent drinkers: >280 g per week (male), >160 (female)Not stated2 BL, 6 months7 Q/F screen drinking diary dependence screen consequences health survey collateral biomarkerAt 6 months: mean units per week (unit = 8 g)[DOWNWARDS ARROW]71, BL 232 (157), F/U 161 (122)−0.31 d = 0.51

The specific design characteristics listed include setting, number of control subjects, inclusion criteria, control group treatment, frequency of assessment and domains measured by assessment instruments. Where published information provided a sufficient level of detail, we calculated an effect size (Cohen's d as the difference in mean consumption from baseline to follow up, divided by the pooled standard deviation for consumption from baseline and follow up). We include here the Crawford study with the caveat that baseline and follow-up measures are not entirely comparable (drinking day vs. drinking session), although the Paddington Alcohol Test was administered at both times [54].

Three studies were conducted in the emergency department, 11 in primary care, and two in a hospital ward/inpatient setting. The sample sizes of these 16 studies ranged from 31 to 370 (mean 134.5, SD 104). Two studies included only high-risk drinkers, while 14 enrolled both high-risk and dependent drinkers, with variations in exclusion of severely dependent drinkers with a history of alcohol withdrawal. Among the 16 studies, six studies offered control subjects only usual care, four gave controls only general alcohol and health information, four provided alcohol-specific feedback/advice to control subjects, and in two the control procedures were not described.

Six studies assessed controls only at enrolment and final follow up, but eight had three assessment time points, one had four and another had seven (median number of assessments = 3). The domains of inquiry and number and type of instruments were also quite different. The majority of the studies assessed outcomes at 12 months, but three had 6 month endpoints and one terminated follow up at 3 months. The range for the percentage of change in consumption measures from baseline to endpoint was from −0.10 to −0.84, with a mean of −0.32. In the 12 studies that had sufficient information to calculate an effect size for these decreases in consumption, Cohen's d ranged from 0.04 to 0.70, with an unweighted mean of 0.37.

Potential sources of control group reductions: the role of bias related to research design

  1. Top of page
  2. Abstract
  3. Introduction and Aims
  4. Reductions in control group alcohol consumption over time
  5. Potential sources of control group reductions: the role of bias related to research design
  6. Studies designed specifically to evaluate assessment reactivity
  7. Strategies to limit inadvertent moderators of control group drinking
  8. Conclusions
  9. References

Several hypotheses have been advanced to account for the phenomenon of control group reductions in consumption. These are presented schematically in Figure 1. As the model demonstrates, a number of design elements have the potential to affect the accuracy of self-report [30] and bias toward the null.

image

Figure 1. Potential mechanisms for change in control groups over time.

Download figure to PowerPoint

Blinding to study purpose (investigation of changes in alcohol consumption and/or associated behaviours)

Lack of blinding of subjects to the purpose of the study may trigger social desirability bias, defined as the inclination of participants to give the researcher what they think the study is designed to find [21,67,68]. As Daeppen [69] says, ‘There is reason to expect that these participants may have been aware of the nature of the study and adjusted their self-report in some manner.’

Sample mix

Enrolment may be affected by design elements, such as reimbursement level or time requirements for completion of study procedures. In a non-treatment population, telegraphing the purpose of the study in consent forms may predispose to enrolling only clients who are already considering change. It is common in the clinical research setting for patients to conceal substance abuse for fear that labelling might affect a practitioner's willingness to provide medical care. Moos [70] cites social control theory as a contributor to this phenomenon, and social learning theory as the context for consequences of non-blinding.

The sample mix for BAI studies is also affected by rate of capture of eligible patients. Refusal to enrol in studies is more common in the primary care setting, where patients are identified by screening, and alcohol use is rarely the presenting problem. A systematic review of lost subjects in BAI studies in general practice [71] describes an attrition rate of 44.3% to 83.2% (mean 70.6%) from the point of eligibility to the point of enrolment. The authors point out that the low participation rate (a mean of 29.4%) may affect generalisability.

In particular, small samples and randomisation without blocking for problem intensity can result in a skewed distribution of acuity. If the sample is weighted toward the low end of the scale (high-risk drinkers), there will be natural fluctuations over time in consumption; if the sample consists of primarily dependent drinkers who consume at outlier levels, there will be a greater tendency to observe variations in measurement as a result of regression to the mean (RTM) [72].

Assessment reactivity

Baseline assessment instruments, such as drinking diaries, inventories of drinking consequences and psychosocial and health scales, are important for evaluating the baseline comparability of intervention and control groups. However, these standardised instruments also have the potential to mimic a BAI. Sobell, who developed the concept of a drinking diary, known as the Timeline Followback (TLFB), herself identifies the potential for assessments of this type to influence control groups [22]. As Moos points out [70], ‘assessment can raise individual's awareness of risky drinking, initiate self-monitoring and lead to recognition of a discrepancy between current behavior and a personal normative standard, . . . communicate implicitly personal responsibility and impel a need for change’, creating bias toward the null [67]. This type of connection is a formal component of motivational interventions to reduce unhealthy alcohol use.

Procedures for control condition discharge

In addition, control groups often receive a combination of normative feedback, advice and referral—all standard elements of brief intervention in the clinical setting. Standards of clinical care require that positive screening results be addressed by practitioners, and in many countries, institutional review boards require that controls receive advice and/or referral resources.

In the interim study period, between enrolment and study discharge, both reactivity to interim assessment and measurement effects from seasonal variation and RTM may play a role in consumption reductions. Interim measurement among subjects enrolled from the clinical setting may also contribute to bias toward the null, as suggested by evidence from a treatment-seeking population, in which the number of intervals and the length of assessment were found to be predictive of consumption outcomes [24,26,73].

Regression to the mean

Alcohol intake varies over time and by season, except in the most dependent of drinkers [74,75]. Drinking diaries focused on a 1 week history may overestimate the number of abstainers [76], enhancing the likelihood of control group reductions. RTM at the time of repeated measurement may be a large factor in self-reported control group declines in alcohol intake. RTM is to be expected, because we are measuring outcome conditions that vary over time, and measuring them with admittedly imperfect instruments; an outlier value at enrolment is likely to be followed by a more moderate value at follow up, and because of this variation, there is likely to be measurement bias, because observations at time one and time two may not reflect a true distribution [72,77].

Measurement/analysis issues

Variance in alcohol consumption endpoints is often large, with skewed distributions or outliers that may reduce statistical power and complicate analyses. The treatment of outliers and transformation of data to account for skewness must be considered for consumption endpoints. Poisson models are problematic for characterising alcohol data, because they may result in a large Type I error [67,78,79]. Failure to adjust for baseline values may also bias toward the null.

Studies designed specifically to evaluate assessment reactivity

  1. Top of page
  2. Abstract
  3. Introduction and Aims
  4. Reductions in control group alcohol consumption over time
  5. Potential sources of control group reductions: the role of bias related to research design
  6. Studies designed specifically to evaluate assessment reactivity
  7. Strategies to limit inadvertent moderators of control group drinking
  8. Conclusions
  9. References

The failure to find group differences in consumption where differences were expected in more than one major, well-designed and well-conducted study [69,80] has given impetus to new initiatives to assess the impact of testing on control groups. We identified six studies in the literature that were designed with the intent to test assessment reactivity in an out-of-treatment population and learn more about the impact of assessment on outcomes after intervention (see Table 2). Two of these, Richmond et al.[63] and Daeppen et al.[69], were randomised controlled trials of BAI in the medical setting that included two control groups, one fully assessed and the other minimally assessed. The other four [80,81,83,84] were trials among college students, in which a fully assessed group was compared with screened-only groups.

Table 2. Comparison of minimal assessment (MA) versus full assessment (FA) groups: Differences in consumption at follow up
Author, year sample size setting, modalityInclusion criteriaMA group proceduresAC assessment procedures #, timing type of duration measurementConsumption measuresOutcome measures at final F/U MA FA mean, SD mean, SDEffect size per published data: Cohen's d (95% CI)
  1. AUDIT, Alcohol Use Disorders Identification Test; BAC, blood alcohol concentration; BL, basline; CI, confidence interval; DDQ, Daily Drinking Questionnaire; GGT, gamma glutamyl transpeptidase test; HED, heavy episodic drinking; NHMRC, National Health and Medical Research Council; RAPI, Rutgers Alcohol Problem Index; TLFB, Timeline Followback.

Walters et al. 2009[81] MA n = 63; FA n = 66 university campus Web-based>18 years High-risk & dependent drinkersOne item: # binge episodes in 2 weeks4 BL, 3, 6, 12 30 min each visit5 7 day diary, AUDIT, norms readiness, strategies for drinking lessAt 12 months: mean drinks per week estimated peak BAC9.98 (7.97) 120 mg/dl (84)9.52 (9.98) 96 mg/dl (75)d = −0.119 (−3.967, 1.953) NS d = −0.373 (−0.055, −0.002) P < 0.05
McCambridge 2009[82] MA n = 144; FA n = 156 university campus face-to-faceAged 18–24 high-risk & dependent drinkersHealth survey; trauma history; one item: injured after drinking2 BL, 2–3 months 2 min5 AUDITAt 2–3 months: AUDIT units per week9.7 (5.5) 18.7 (20.4)8.3 (6.0) 15.9 (19.6)d = 0.23 (0.01,0.45) P < 0.05 d = 0.13 (−0.09,–0.36) NS
Daeppen et al. 2007[69] MA 257; FA 277 emergency department face to faceAged >18 injured high-risk & dependent drinkersHealth survey with embedded Q/F screen2 BL. 12 months 30 min4 health survey; AUDIT 7 day diary SF-12At 12 months: # drinks/7d #HED per month AUDITBL no data F/U 10.9 14.2) BL 3.7 (6.1) F/U 3.6 (6.4) BL no data F/U7.3 (4.7)BL 13.3 (14.7) F/U 11.1 (11.9) BL 4 (6.2) F/U 3.6 (6.1) BL 8.7 (4.1) F/U 7.2 (4.3)NS NS NS
Kypri et al. 2007[83] MA n = 126; FA n = 126 student health clinic Web baseAged 17–29 AUDIT >8 max drinks duration of episodeAUDIT alcohol use survey; and alc facts &effects leaflet only 3.3 min4 BL, 4 weeks, 6,12 10 min5 14 day diary problems, norms, academic role expectationsAt 12 months: # drinks in 2 weeks #days HED per 2 weeks AUDITMedian (range) 30 (0–175) 1(0–8) BL 15 (SD 5.4) F/U 14 (2–30)Median (range) 25 (0–168) 0 (0–8) BL 14.9 (SD 5.0) F/U 13 (19–29)0.82 (0.68,0.98) P = 0.03 0.66 (0.47, 0.91) P = 0.01 −1.63 (−2.65, −0.62) P < 0.001
Carey et al. 2006 [80] MA 59; FA 72 university campus face to faceAged 18–25 >1 HED per weekDDQ Q/F calc BAC problems RAPI collaterals4 BL, 1,6,1290 day diary drug and sex TLFB video/audio tapeAt 12 months: drinks per week # days HEDMean (SD) BL 19.4 (12.4) F/U 15 (10.5) BL 7.7 (4.1) F/U 5.1 (4.0)Mean (SD) BL 18.1 (8.9) F/U 16.2 (11.6) BL 6.8 (3.8) F/U 6.3 (4.3)Published data not available to calculate statistical differences between MA (control) and FA (TLFB) groups
Richmond et al. 1995 [63] MA 72; FA 66 general practice face to face not randomly assigned (allotted randomly to one of four groups with blocks determined by GPAged 18–70 M > 35 per week F > 21 per week MAST > 20 Ph > 10 excluded severe dependenceHealth and Fitness Questions Q/F BL and 6 month; then self help manual3 BL 6, 127 day diary MAST/Ph Comp Drinker Profile Blood/GGTAt 6 months: 7d consumption drank below NHMRC levelsBL no data F/U 33.8(29.9) no data F/U 69.9%BL 32.5 (27.7) F/U 27.6 (26.7) BL 73.1% F/U 71%Published data not available to calculate statistical differences between MA (control) and FA (TLFB) groups

Among these trials, only McCambridge and Day [84] attempted to blind subjects to the study purpose. Fully assessed controls completed one assessment instrument at baseline, the Alcohol Use Disorders Identification Test (AUDIT) 10, but the minimally assessed control group received only a baseline health survey that included a single alcohol question (ever alcohol in conjunction with injury). Both groups were assessed with the AUDIT 10 at the end of a 2–3 month follow-up period. The fully assessed group reduced their AUDIT score by 1.4 points more than the minimally assessed group, an effect size of 0.23.

Kypri et al.[83] conducted a study among a sample recruited from a student health centre roster, which is assumed to have higher acuity than the general college population. This study also utilised the AUDIT as an outcome measure. Minimally assessed controls received only the AUDIT at baseline, while fully assessed controls received more frequent and more extensive assessment (drinking diary, alcohol problems, academic impact and norms). At 12 months, the fully assessed controls reported a 1.6 point greater reduction in mean AUDIT scores.

Walters et al.'s[81] Web-based campus study showed no difference in mean drinks per week between minimally assessed controls (one question about binge episodes) and fully assessed controls (7 day diary, AUDIT, norms, readiness and strategies for drinking less), but did report a difference in mean blood alcohol concentration (an effect size of 0.33). For our analysis (Table 2), we did not use their AUDIT results as an outcome measure for consumption, because the AUDIT score is composed of both consumption and consequences and it is not possible to isolate out quantity and frequency from these data.

In the other campus study by Carey et al.[80], conducted face-to-face, the minimally assessed control group actually had greater decreases in drinking outcomes baseline to 12 months than the fully assessed control group (TLFB plus). In all four college studies, the minimally assessed group scores decreased over time. Not all baseline data were available to determine the extent of improvement in these studies.

Interestingly, Daeppen's study among injured emergency department patients [69] failed to show a dose–response curve (a pattern of smaller to larger decreases from minimal to full assessment conditions to intervention) or any difference between minimally assessed controls (screened only with quantity/frequency questions and a health history) and the fully assessed controls (who completed the AUDIT, the TLFB and the SF-12) at the 12 month follow up. Although the fully assessed controls reduced consumption over baseline; no baseline data were available to assess changes in the minimally assessed group. Daeppen attributes his findings of no BAI effect to reporting bias associated with a lack of blinding to the purpose of the study, to RTM and to the impact of the injury itself.

Richmond in 1995 [63] was one of the first investigators to design a specific control for assessment reactivity as part of a trial of a five-session Alcoholscreen program in general practice. She found no assessment reactivity at the 6 month follow up between those who were screened only and a group that received screening and the TLFB. Baseline data were not collected on the screened only, but at follow up the screened only were drinking at higher levels than the Timeline group (33.8 vs. 27.6). A decline of five drinks over 7 days of drinking was observed for the TLFB only group, but there was no way to compare percentage change with the screened-only group.

The studies conducted with college students suggest the possibility that minimal brief assessment with validated and reliable instruments might have a mild intervention effect, but results were mixed. If there were such an effect, electronic administration to large populations (patient rosters and across college campuses) would have the potential for a significant public health impact. Neither the Daeppen nor the Richmond study found a difference between minimally and fully assessed control groups, nor did Ogborne and Annis[85] in a treatment setting. We are a long way from having the evidence to determine if brief screening alone might be valuable in specific settings or for sub-populations. The level of the evidence that we do have at this point in time demands caution in extrapolating and translating findings.

Strategies to limit inadvertent moderators of control group drinking

  1. Top of page
  2. Abstract
  3. Introduction and Aims
  4. Reductions in control group alcohol consumption over time
  5. Potential sources of control group reductions: the role of bias related to research design
  6. Studies designed specifically to evaluate assessment reactivity
  7. Strategies to limit inadvertent moderators of control group drinking
  8. Conclusions
  9. References

A variety of strategies have been recommended to decrease the impact of research design features on control group drinking. There is general agreement on the following areas. Assessment reactivity can be minimised by reducing the burden of assessment for control groups, decreasing the number of assessments as well as the scope. Kypri [67] suggests utilising briefer versions of standardised instruments, that is replacing the 10-question AUDIT with the also validated AUDIT C, which has three questions. Others [26,70,82,86] call for studies that employ the addition of a minimal or delayed assessment control group in order to begin to define best practices. Nye et al.[68] suggests that normative feedback be avoided, so as not to create heightened awareness that mimics intervention. Kypri [67] would also suggest eliminating readiness scales for the same reason. Daeppen et al.[69] emphasises the importance of subject blinding to the purpose of the study. Kypri [67] proposes care in assessing power requirements, addition of biomarkers to enhance veracity, and attention to randomisation procedures that will ensure a wider range of subjects and thus limit potential for bias, and Morton [72] stresses the importance of avoiding selection of the most severe cases in order to avoid pitfalls associated with RTM. Edwards and Rollnick [71] point out that better management of study process might enhance the number of eligible patients who actually enrol, increasing sample generalisability. Block randomisation to capture a range of acuity might also be useful in this regard. If standardised definitions were used for drinking variables, it would be easier to compare outcomes across studies. There are also excellent resources for techniques to enhance follow-up rates and thus limit missing data [87]. The potentially negative effects of multiple assessment points must be balanced against the positive value of capturing information across temporal variations. And finally, use of appropriate analytic techniques, such as the negative binomial distribution, may limit Type I errors. The use of ancova[88] and basing selection on more than one measurement value [89] are both recommended to limit RTM.

It is important to note that published data do not offer sufficient information for meta-analysis of changes in individual consumption over time. Jenkins et al. recognise the problems inherent in a categorical approach to population data in a recent letter to Drug and Alcohol Dependence[90]. In our analysis of control group change in alcohol consumption over time (see Table 1), we found design conditions for control groups in these BAI studies to be highly variable, with considerable differences in screening procedures, number of assessment intervals, number of testing points, number of instruments used and number and type of domains of testing. While we would have liked to perform a meta-regression to evaluate the contribution of these potential predictors of change, most of the published reports included in our review lack information needed for this type of analysis. These studies generally report standard deviations for alcohol consumption at baseline and again at follow up, but do not provide a standard deviation for the change in consumption that is needed for a formal meta-regression of study factors affecting change in consumption. The inability to evaluate predictors of control group change because of data missing from published accounts of BAI investigations and lack of common terms and procedures is a serious limitation to our understanding of mechanisms of action; it underscores the importance of developing methods for identifying and adjusting for potential sources of bias prospectively.

Conclusions

  1. Top of page
  2. Abstract
  3. Introduction and Aims
  4. Reductions in control group alcohol consumption over time
  5. Potential sources of control group reductions: the role of bias related to research design
  6. Studies designed specifically to evaluate assessment reactivity
  7. Strategies to limit inadvertent moderators of control group drinking
  8. Conclusions
  9. References

Control groups in trials of BAI in the clinical setting reduce drinking significantly over time. Study design elements that contribute to this decrease may mask the effectiveness of intervention by biasing toward the null. Studies specifically designed to evaluate factors related to assessment reactivity and other design issues are now beginning to be reported and are likely to demonstrate small but significant effects of assessment on control group participants. This review underscores the need to design prospective studies that permit the identification of and adjustment for potential sources of bias. Strategies available to reduce the impact of moderators on control group consumption include procedures for sample size determination and selection, reducing the burden of assessment, and analytic techniques to decrease the impact of RTM.

References

  1. Top of page
  2. Abstract
  3. Introduction and Aims
  4. Reductions in control group alcohol consumption over time
  5. Potential sources of control group reductions: the role of bias related to research design
  6. Studies designed specifically to evaluate assessment reactivity
  7. Strategies to limit inadvertent moderators of control group drinking
  8. Conclusions
  9. References