As Mark Twain remarked, ‘It ain't so much the things we don't know that get us in trouble. It's the things we do know that ain't so.’ Systematic reviews are intended to address just this problem. By transparently and systematically applying explicit methods to all empirical evidence that meets eligibility criteria, the review is designed to provide reliable findings that discriminate real truths from spurious ones . Our systematic review  evaluated the presumption in some clinical practice guidelines that trying to manage weight while quitting smoking is likely to undermine tobacco abstinence. The review produced no evidence that combining behavioral treatments to address smoking and weight undermines smoking cessation. In fact we found the opposite: that combined treatment tends to improve both smoking and weight outcomes. By dispelling a false worry, these findings help providers to assist the many smokers who decline to try to quit unless given a means to address weight gain.
We appreciate Parsons et al.'s response  to our prior comments , and perceive that our groups agree about more than we disagree. One apparent point of consensus is that using more conservative random-effects analysis, as we did in our review , rather than fixed effects is preferable when heterogeneity is present and when there is intention to generalize beyond the specific studies included in the review. A second apparent point of agreement is that including different randomized controlled trials (RCTs), as our two reviews did, is often a reason why systematic reviews reach different conclusions.
On the other hand, we disagree that there is lack of consensus about whether to apply intent-to-treat (ITT) analysis in RCTs of behavioral interventions. The intent-to-treat principle specifies analyzing all participants in the condition to which they were randomized regardless of whether they received or completed the assigned treatment. ITT is universally endorsed because alternative analytic approaches depart from randomization and invalidate trial interpretation [5, 6]. Using ITT analysis is not only feasible, it is essential if we hope to equip policy makers to act on the results of behavioral clinical trials.
Implementing ITT policy (‘once randomized, always analyzed’) is most straightforward when there are few or no missing outcome data. Parsons et al. suggest that few investigators will perform the ‘heroics’ needed to collect outcome data from most of the originally randomized sample. In contrast, we believe that a growing number of behavioral clinical trialists appreciate the need for complete follow-up. Investigators who both conduct RCTs and synthesize them for systematic reviews are especially likely to have learned this lesson, since 85–90% follow-up is a quality criterion in several clinical trial rating scales [7, 8]. Parsons et al. infer that investigators will continue to assess smoking and weight outcomes only from adherent, smoke-free study participants who attend clinic visits to receive treatment. We think those days are in the past. We find current trialists more likely to adopt better behavioral clinical trials practice. For example, many perform outcome assessments separately from treatment visits in order to keep outcome assessors blinded to treatment assignment [9–11].
We also doubt that researchers will continue to use over-deterministic imputation procedures to estimate missing data. It has become well-understood that trial results can be biased, and not necessarily in a conservative direction, by imputing smoking for missing tobacco status data or by carrying forward baseline or last weight for missing weight status data [12–14]. In the future, we expect even more behavioral trialists to apply appropriate multiple imputation or maximum likelihood approaches to estimate missing data [15, 16].
In the final analysis, the function of a systematic review is to replace assumptions with conclusions drawn from unbiased evaluation of data. Despite the systematic review's many methodologic protections, bias can enter if too many presuppositions drive the conduct of the review. We find a few assumptions problematic in Parsons et al.: (i) ‘An intervention that increases abstinence will increase weight gain.’ The data on combined smoking and weight interventions suggest otherwise . (ii) ‘Introducing a very low calorie diet (VLCD) together with smoking cessation will reduce weight gain because it reduces hunger.’ Combining a VLCD and cessation treatment has been found to increase hunger . (iii) Weight gain is not likely to be a problem for smokers who quit and then resume smoking because ‘smokers who relapse lose the weight they gained.’ Even after 8 years, smokers who had been abstinent for a year and then resumed smoking show somewhat greater weight gain than continuing smokers, suggesting that weight gain was greater still and provided even more cause for concern earlier in the cessation process .
We thank Parsons et al. for initiating this dialogue to upgrade the quality of RCTs and systematic reviews in the addictions. In support of that goal, we call readers' attention to online learning modules about these methodologies. That resource, sponsored by the US National Institutes of Health, can be accessed free of charge or for continuing education credit by registering at http://www.ebbp.org/training.html