• Meta-analysis;
  • online health promotion;
  • smoking cessation;
  • systematic review;
  • web-based intervention


  1. Top of page
  7. Declarations of interest
  8. Acknowledgements
  9. References

Aim  To examine the efficacy and acceptability of online, interactive interventions for smoking cessation and to identify treatment effect moderators and mediators.

Methods  A systematic review and meta-analysis of the literature (1990–2008) was conducted, finding 11 relevant randomized controlled trials. Data were extracted and risk ratios and risk differences estimated with a random effects model.

Results  There was no evidence of publication bias. Included trials were of variable methodological quality. Web-based, tailored, interactive smoking cessation interventions were effective compared with untailored booklet or e-mail interventions [rate ratio (RR) 1.8; 95% confidence interval (CI) 1.4–2.3] increasing 6-month abstinence by 17% (95% CI 12–21%). No overall effect of interactive compared with static web-based interventions was detected but there was significant heterogeneity, with one study obtaining a clear effect and another failing to find one. Few moderating or mediating factors were evaluated in studies and those that were had little effect. Pooled results suggest that only interventions aimed at smokers motivated to quit were effective (RR 1.3, 95% CI 1.0–1.7). Fully automated interventions increased smoking cessation rates (RR 1.4, 95% CI 1.0–2.0), but evidence was less clear-cut for non-automated interventions. Overall, the web-based interventions evaluated were considered to be acceptable and user satisfaction was generally high.

Conclusion  Interactive, web-based interventions for smoking cessation can be effective in aiding cessation. More research is needed to evaluate the relative efficacy of interactive web-based interventions compared with static websites.


  1. Top of page
  7. Declarations of interest
  8. Acknowledgements
  9. References

Smoking is one of the most significant factors contributing to low life expectancy, health inequalities and ill health world-wide [1]. Smoking prevalence has fallen considerably, especially in developed countries, and yet fewer than half of smokers ever achieve long-term abstinence despite 70–80% wanting to quit and one-third having made at least three serious quit attempts [2]. This underlines the need for the development of new methods that prevent people from starting to smoke, motivate smokers to quit smoking and sustain long-term cessation.

Effective smoking cessation interventions exist [3]. However, the most effective interventions (such as face-to-face counselling) also have the lowest reach and interventions with the widest reach (e.g. mass media campaigns) have low efficacy [4]. One way forward is to either increase the reach of intensive, low-reach interventions or the efficacy of less intensive, high-reach interventions [5]. Tailoring of information has been shown to increase effectiveness of low-intensity interventions such as self-help materials [6], but these do not allow smokers to interact as they can in face-to-face interventions. With advances in computer-based communication technologies, the internet now offers the potential to combine the efficacy of intensive treatments with the advantages of wide-reaching interventions through incorporating interactive responses [7], thus adopting some of the features of face-to-face contact [8]. Moreover, internet-based interventions can exploit economies of scale and are therefore potentially very cost-effective [9].

Currently, around 75% of the population in industrialized countries and around 10% in the rest of the world have access to the internet [10,11], and these proportions are likely to rise further. Online support for smoking cessation therefore represents a viable resource for smokers wanting to stop, especially because it can be accessed at any time from anywhere with an internet connection [12]. However, studies of online smoking cessation resources find great variability in quality and acceptability: up to 80% of reviewed websites provided no information on key components endorsed by smoking cessation clinical guidelines [12]. Consequently, many users report being unsatisfied with web-based smoking cessation interventions, finding them confusing and unhelpful [13]. Moreover, despite an increase in the number of online interventions, relatively little is known as to how and why these interventions work and which factors influence their effectiveness [14].

Several reviews have evaluated the use of internet-based interventions to change health behaviours, including physical exercise, weight loss, disease management and prevention (see [15–19]). While treatment efficacy differs with the health behaviour that is targeted, these reviews generally support the usefulness of web-based interventions for health promotion. Two reviews that focused exclusively on smoking cessation interventions [20,21] concluded that computer-based, including web-based, interventions have the potential to increase smoking cessation rates. However, both reviews included interventions that, although computer-based, were not interactive or delivered online.

The delivery and content of websites is not regulated [22], and it is easy to imagine how variability across websites could make generalizations about this type of intervention problematic. However, it is possible to assess the potential for this kind of intervention to provide smoking cessation support by examining the efficacy of rigorous evaluations carried out to date. To our knowledge, this is the first meta-analysis to evaluate the efficacy of interactive, online smoking cessation interventions compared with either low-efficacy interventions with high potential reach (e.g. self-help materials such as booklets) or with high-efficacy interventions of low potential reach (e.g. face-to-face counselling). This review also includes studies not covered in other published reviews. Moreover, we aimed to clarify possible mechanisms of action by identifying potential moderators and mediators (i.e. specific intervention components) of the treatment effect.


  1. Top of page
  7. Declarations of interest
  8. Acknowledgements
  9. References

Study selection

To be included in this review articles had to be in English, published since 1990, have a randomized controlled trial (RCT) design with at least 1 month follow-up assessing smoking cessation and participants had to be current smokers at enrolment. In order to differentiate the potential of internet-based interventions from traditional self-help material, only interventions that made use of the interactive nature of the internet were included. Multiple follow-up studies that used data from the same cohort of participants were excluded to avoid bias in effect estimates [23]. Studies providing insufficient detail/data and those focusing on secondary outcomes (such as training health professionals involved in providing smoking cessation interventions) were also excluded.

Data source and search strategies

Relevant literature was sought by searches of computerised databases carried out in December 2008. The electronic resources used were PubMed (1990–2008/12), PsycINFO (1990–2008/12), CINHAL Plus (1990–2008), EconLit (1990–2008/11), ISI Web of Science® (SCI, SSCI, AHCI; 1990–2008) and CENTRAL (Cochrane Registry of Controlled Trials, 1990–2008). Medline was searched with relevant MeSH terms {[(‘Online Systems’ OR ‘Internet’) AND ‘Smoking Cessation’] AND ‘Randomized Controlled Trial’}. All other databases were searched with free text terms reflecting inclusion criteria: {[(‘Internet’ OR ‘Online’ OR ‘Web’) AND ‘Smoking cessation’] AND (‘RCT’ OR ‘Randomi$ed controlled trial’ OR ‘Random* trial’)]}. This yielded 82 studies, reduced to 14 studies when reading the abstracts and nine suitable studies after scrutinizing the full text of papers. Further free text and bibliography searches revealed two additional papers that fulfilled selection criteria, bringing the total of included studies to 11 [24–34]. Egger's regression intercept, as well as the Begg & Madzumdar rank correlation coefficient, which are used to quantify publication bias [35], were non-significant, suggesting that no file drawer effect was present.

Data extraction and analysis

Data were extracted manually and recorded on a data extraction sheet to minimize bias when coding and rating studies [36]. Results were analysed with RevMan [37] and presented as rate ratios (RRs) and, where appropriate, risk difference was calculated. The extent of heterogeneity between study outcomes was assessed using the I2 statistic, which is based on Cochrane's Q but is more robust, and thus suitable for smaller meta-analyses [38]. Where appropriate, studies were pooled with the Mantel–Haenzel method using random effect rather than fixed effect models to maximize accuracy of parameter estimation [39].

Study quality

Attrition rates tend to be high in internet-based interventions [40]. Indeed, attrition in studies varied widely (Table 1), suggesting that some of the results would be affected by attrition bias. All studies (except [32]) therefore used an intention-to-treat analysis. The methods of recruitment and the populations sampled were reasonably homogeneous (Table 2). However, little information was given about participants included and excluded from studies, making it impossible to evaluate the representativeness of the included samples. Studies also provided little detail on intervention characteristics and whether outcome assessors were blinded.

Table 1.  Study methodology.
ReferenceTrue rando-mizationaMatched group characteristicsMatched treatmentStandardized proceduresTrained staffMeasure and outcome validationBlinded assessorAttrition rate(%)ITT analysisPower and statistical analyisis
  • a

    The use of alternation, case record numbers, birth dates or weekdays is considered to be inadequate for randomization (Verhagen, de Vet, de Bie, Kessels et al., 1998);

  • Attrition rate refers to cumulative loss to follow-up at last contact point; C: control condition; I: intervention; ITT: intention to treat; GEE: generalized estimating equation; NA: not available; CO: carbon dioxide.

An et al.[24]YesIntervention group older, no differences on other measuresMinimalYesNot knownMeasures validated but abstinence only partly confirmed with biomarker (CO); 30 day abstinenceNot knownC: 8.8 I: 7.0Yesβ = 0.85 to detect 12% difference; little detail on statistics: mostly logistic regression; statistical assumptions not tested
Brendryen & Kraft [25]YesNo difference on small set of measuresSomewhatYesNAMeasures validated except self-efficacy measure; abstinence not biochemically validated; 7 day point prevalenceNot knownC: 15.6 I: 12.2YesPowered but not specified; detailed statistics: χ2, t-test, logistic regression; statistical assumptions not tested
Brendryen et al.[26]YesNo differences on extensive set of measuresSomewhatYesNAMeasures validated except self-efficacy measure; abstinence not biochemically validated; Repeated 7-day point abstinenceNot knownC: 26.0 I: 18.1YesPower unclear; detailed statistics: χ2, t-test, logistic regression; statistical assumptions not tested
Japuntich et al.[27]UnclearNo test of differencesMinimalYesYesMeasures validated; abstinence validation unclear; 7 day point prevalenceNot knownC: 20.8 I: 19.3YesPower unclear; little detail on statistics: mostly logistic regression
McKay et al.[28]UnclearNo differences on extensive set of measuresYesYesNot knownMeasures validated except self-efficacy, and depression abstinence not biochemically validated; 7 day point prevalenceNot knownC: 72.6 I: 72.9YesPowered but not specified; detailed statistics: logistic regression, Kaplan–Meier survival analysis; statistical assumptions not tested
Munoz et al.[29]UnclearNo test of differencesYesYesNAMeasures validated, abstinence not biochemically confirmed; 7 day point prevalenceNot knownC: 52.6 I: 50.9YesPower unclear; little detail on stats: χ2 test
Patten et al.[30]UnclearNo differences on extensive set of measuresYesYesYesMeasures validated; cessation confirmed by CO; 30 day abstinenceNot knownC: 48 I: 41YesPower unclear; detailed statistics: χ2, Fisher's exact test, rank sum test, GEE
Pike et al.[31]UnclearNo differences between groups on unknown set of measuresMinimalYesNot knownNo validation of measures provided; abstinence not biochemically confirmed; 7 day point prevalenceNot known46—similar in both groupsYesβ = 0.8 to detect 3% difference; little detail on statistics: χ2 and t-test, statistical assumptions not tested
Prochaska et al.[32]UnclearCannot be estimatedYesYesYesMeasures, validated, abstinence not biochemically confirmed; Point prevalenceNot known25.7—not provided by groupNoPower unclear; little detail on statistics: χ2 and Levy test for pairwise proportions
Strecher et al.[33]UnclearNo differences between groups on unknown set of measuresSomewhatYesNAMeasure validation not known, abstinence not biochemically confirmed; 10-week abstinenceNot knownC: 78.9 I: 77.6YesPower unclear; little detail on statistics: logistic regression, statistical assumptions not tested
Swartz et al.[34]YesNo differences between groups on extensive set of measuresNot at allYesNAMeasure validation unclear, abstinence not biochemically confirmed; 7-day point prevalenceNot knownC: 38.1 I: 49.1YesPower unclear; detailed statistics: χ2 and logistic regression
Table 2.  Study summary.
ReferenceSample and settingRecruitmentInterventionFollow-upDesignMain findings
  1. NRT: nicotine replacement therapy; Q&A: question and answer; RCT: randomized controlled trial; ROI: Republic of Ireland.

An et al.[24]517 college smokers at University of MN, USA; not required to intend to stop smokingCollege internet health screening and follow-up e-mailRealU: website based on social cognitive and problem behaviour therapy; logs health/life-style habits, has interactive quiz and feedback, links to life-style magazine, provision of weekly, tailored e-mails from peer coaches; provision of financial incentive for participation8, 20, 30 weeksRCT-2 groups: (i) control condition (e-mail with link to online health and academic resources); (ii) RealU interventionParticipants in (ii) more likely to report having stopped smoking for a month at follow-up
Brendryen & Kraft [25]396 smokers recruited from general population; Norway; required to intend to stop smokingBanner advertisement in internet newspaperHappy Ending: daily e-mail to direct to webpage; pre-recorded daily audio message (log-on call) and up to three text messages, proactive log-off call in evening to assess smoking and provide relapse prevention information when necessary, plus directs to craving helpline; fully automated tunnel design; participants received reminders1, 3, 6, 12 monthsRCT-2 groups: (i) booklet intervention (booklet with information, quit calendar, telephone number and links to online resources + NRT sample); (ii) Happy Ending intervention + NRT sampleParticipants in (ii) significantly more likely to have stopped smoking and to adhere to NRT
Brendryen et al.[26]290 smokers recruited from general population; Norway; required to intend to stop smokingBanner advertisement in internet newspaperHappy Ending: daily e-mail to direct to webpage; pre-recorded daily audio message (log-on call) and up to three text messages, proactive log-off call in evening to assess smoking and provide relapse prevention information when necessary, plus directs to craving helpline; fully automated tunnel design; participants received reminders1, 3, 6, 12 monthsRCT-2 groups: (i) booklet intervention (booklet with information, quit calendar, telephone number and links to online resources); (ii) Happy Ending interventionParticipants in (ii) significantly more likely to have stopped smoking and to have higher levels of coping planning and pre-cessation self-efficacy
Japuntich et al.[27]284 smokers recruited from general population in WI, USA; required to intend to stop smokingBillboards, posters, fliers, TV advertisement, press releasesCHESS SCRP: online system with log in leading to tailored feedback, information provision, support centre providing web-based cognitive–behavioural techniques (chatroom/ask the expert) and folder to carry out emotional writing paradigm/keep diary; participants received reminders3, 6 monthsRCT-2 groups: (i) standard intervention: 9 weeks buproprion, three brief counselling sessions and five follow-up visits; (ii) as (i) plus CHESS SCRPNo difference in cessation rates between (i) and (ii) but actual use of online system predicted cessation
McKay et al.[28]2318 smokers recruited from general population in USA; required to intend to stop smoking smokingBanner advertisement on internet search enginesQSN: tailored (tunnel design), interactive webpage providing educational information, cessation preparation, personal quit plan and date, behavioural tasks, thought and mood management, peer support (chatroom), expert advice and prompts3, 6 monthsRCT-2 groups: (i) Active Lives: online personalized fitness programme to encourage cessation comprising multi-step plan with motivational component, action plan with tracking, online resources and peer support (chatroom); (ii) QSNNo differences in abstinence rates between (i) and (ii) due possibly to low engagement with online programmes
Munoz et al.[29]568 English- or Spanish-speaking smokers recruited from general population in USA; required to intend to stop smokingPress release and internetGuia + itemized encouraging reminder e-mails (ITEMS) + mood management (MM): web-based intervention of brochure (Guia) providing standard cessation information, tailored advice; ITEMS; online (MM) course providing instructions about monitoring feelings and relaxation; fully automated; financial incentive1, 3, 6, 12 monthsRCT-2 groups: (i) Guia + ITEMS; (ii) Guia + ITEMS + MM Carried out either as integrated in Spanish or English websiteIn separate analysis for Spanish and English websites studies, lower abstinence rates in (ii) than (i) for latter website
Patten et al.[30]139 adolescent smokers from MN, WI, CT, USA; not required to intend to stop smokingTV, radio newspaper advertisement, fliers in schools and health clinicsSOS: web-based with 38 components including interactive services (quizzes, tailored quit plans and notes, Q&A e-mail service, online discussion support group), information services and videos (showing personal stories of quitters); financial incentive8, 12, 24 (36) weeksRCT-2 groups: (i) brief office intervention (four sessions of motivational interviewing/information and homework); (ii) SOS interventionNo difference in cessation rates between (i) and (ii) but decrease in days smoked in (i)
Pike et al.[31]6451 smokers recruited from general population; USA; not required to intend to stop smokingAdvertisement on American Cancer Society websiteFive websites (QuitNet, Smoke Clinic, V-CC, ProChange, Oregon Center for Applied Sciences) providing tailored, interactive service including peer support, role models, e-mails, expert advice, information provision and chatroom3 monthsRCT-2 groups: (i) access to static internet site containing downloadable self-help booklets; (ii) access to intervention websites free of chargeNo difference in cessation rates between (i) and (ii) but in (ii) greater quit rates for those assigned to highly utilized websites
Prochaska et al.[32]136 smokers employed at university; USA; not required to intend to stop smokingMail, e-mail, telephone callsProChange LifeStyle: multi-behaviour targeted online intervention programme involving assessment and tailored feedback on basis of stage of change model; fully automated6 monthsRCT-3 groups: (i) brief health risk intervention (based on stages of change);(ii) as (i) + three face-to-face motivational interviewing sessions (iii) as (i) + ProChange interventionNo difference between groups
Strecher et al.[33]3971 smokers from general population UK and ROI; required to intend to stop smokingInternet website provided in support material of NRT purchased over the counterCQ Plan: initial assessment, tailored newsletters, support messages sent by e-mail (based on cognitive–behavioural methods); identification of friend who received tailored e-mail advice to support quitter; fully automated6, 12 weeksRCT-2 groups: (i) web-based control condition that provided similar information but was not tailored; (ii) CQ Plan interventionSignificantly greater abstinence rates in (ii) than (i) at follow-up
Swartz et al.[34]351 smokers recruited through worksites; USA; required to intend to stop smokingPosters, brochures, work intranet website, e-mail and newsletter1–2–3 Smokefree: demographically personalized programme providing information, videos to simulate social modelling, tailored quit plan and motivational tips; fully automated90 daysRCT-2 groups: (i) waitlist control (ii) 1–2–3 Smokefree interventionSignificantly greater abstinence rates in (ii) than (i) at follow-up

Most studies (n = 10) used well-validated questionnaires to measure constructs of interest. However, validation of smoking cessation outcome was limited: only two studies verified smoking status objectively, either in a subsample [24] or in the total sample [30]; the remainder relied on self-report. There were inconsistencies in the definition of abstinence between studies; 7- or 30-day point prevalence and continuous abstinence were used in different studies.

Only one study [24] detected baseline differences between groups, but information on socio-demographic and smoking characteristics was lacking for a number of study samples (Table 3). Treatment and control group content were often not well matched (Tables 1–3). This imbalance between control and intervention conditions makes it harder to evaluate whether differences between groups were due to the intervention or confounding variables such as direct contact time.

Table 3.  Study samples and treatment characteristics.
ReferencePopulation demographicsaTreatment
Age (years)Female (%)White (%)High school educationb (%)Smoking (cigarettes/day)LengthIntervention complexitycControl complexityd
  • a

    Where necessary computed indirectly from provided data.

  • b

    b Schooling until at least 14 years of age.

  • c

    c Refers to number of materials used in the intervention, the more complex, the more materials used (e.g. chatrooms, tailored feedback, quizzes, etc.).

  • d

    d Level of sophistication of treatment in control relative to intervention group (NB: in cases of more than two study groups, control group refers to group most closely matching intervention group).

  • e

    e Data come from subgroup in study for which no separate characteristics were provided.

An et al.[24]2072.91001.920 weeksHighLow
Brendryen & Kraft [25]36.250.318.21 yearHighLow
Brendryen et al.[26]39.65017.11 yearHighLow
Japuntich et al.[27]40.854.979.196.821.690 daysHighMedium
McKay et al.[28]∼4270.586.693.1∼21Not providedHighHigh
Munoz et al.[29]36.754.469Unclear21.68 weeksHighHigh
Patten et al.[30]15.749.688829.96 monthsHighHigh
Pike et al.[31]417021Not providedHighLow
Prochaskaeet al.[32]6 monthsHighHigh
Strecher et al.[33]36.956.523.510 weeksHighMedium
Swartz et al.[34]∼4251.982.1∼19Not providedHigh

Details on statistical procedures were often missing (Table 1); just two studies [24,31] reported use of power calculations to estimate the required sample size before recruitment, and one of these postulated an optimistic group difference of 12% [24]. A realistic treatment effect size of 2–5% [i.e. a small effect (Cohen's w = 0.1)] would require a minimum sample of 393 participants per group to detect this difference at a significance level of 5% with a power of 80%. Only three studies in this review [28,31,33] reached this number of participants, while two studies [30,32] were in danger of Type II error because they were severely underpowered (n < 100 per group).


  1. Top of page
  7. Declarations of interest
  8. Acknowledgements
  9. References

Study description

All 11 reviewed studies were published since 2005, included a total of 15 511 participants, and most (n = 8) were carried out in the United States (Table 2). Participants were recruited largely from the general population, with the exception of two studies [24,30] that targeted college students and adolescents. While most trials required participants to intend to stop smoking to be eligible, four studies did not state explicitly or require that smokers had to be willing to stop (Table 2).

Studies included mainly white, female smokers with at least high school education of a wide age range (Table 3). There was great variability in cigarette consumption, as some studies (notably [24]) included non-daily smokers; however, smokers in most studies consumed at least 10 cigarettes a day.

The majority of RCTs compared an interactive internet-based intervention with either a minimal control condition (booklet or a static website containing information derived from standard smoking cessation material) or with a waiting-list control (Table 2). Two studies compared an interactive, internet-based intervention with face-to-face counselling [30,32], and two RCTs ([27,32]; [32] had two control conditions) evaluated the efficacy of an internet intervention as an adjuvant to a behavioural intervention compared with the behavioural intervention alone. Two further studies contrasted an interactive web-based control intervention with an interactive online intervention that was ‘enhanced’ by either including more targeted, comprehensive material [28] or by having additional treatment components [29] (Table 2).

As a minimum, all interventions included baseline and follow-up assessments to tailor provision of information, structure quit plans and deliver relapse prevention advice. Tailoring (the iterative and ipsative use of input information to structure output) is only one way to make use of the inherent interactive nature of the internet; most studies (n = 8) also employed e-mails and/or text messages to contact participants periodically and to provide support material. In addition, they supplied links to other websites with further information and used quizzes to consolidate disseminated information. Four studies used reminder telephone calls or e-mails if participants had not been in contact for a predetermined period of time (Table 2).

Six interventions were fully automated; the remainder contained some opportunity for smokers to interact with ‘peer coaches’ or ‘experts’ to ask for advice, as well as allowing participants to interact with peers via chatrooms (Table 2). Three interventions [30,31,34] featured role models, mainly through the use of videos that were matched to smokers' characteristics, with the aim of increasing motivation and self-efficacy through social modelling; Brendryen et al.[25,26] provided an automated craving line that participants could call if they felt the urge to smoke. Two studies [25,27] provided concomitant pharmacotherapy while one study [33] included only participants who had purchased nicotine replacement therapy (NRT) over the counter.

The majority of intervention programmes were based loosely on evidence-based practice guidelines [41], social cognitive theory [42], emotional writing paradigm [43] and the Stages of Change model [44]. Overall, the length of treatment varied considerably from 10 weeks [33] to 1 year [25], as did follow-up duration (Tables 2 and 3).

There was variability in the control conditions provided (Table 3). Only four studies compared the web-based intervention with an equivalently complex control intervention [28–30,32]. A number of studies attempted to mimic the intervention in the control condition by providing a static website containing general information about smoking and smoking cessation based on standard smoking cessation manuals and booklets (Table 2). Two studies offered only minimal [24] or no control intervention [34].

Study outcomes

Smoking cessation

At the final follow-up point, five studies [24–26,33,34] found an overall positive effect of the intervention on smoking cessation rates. Three further studies found an effect in subgroup analyses [27,30,31] and three studies did not detect a positive effect of the intervention [28,29,32]. As study methodology and intervention content were diverse, we did not pool study results overall. However, in order to present some meaningful data and quantify individual results, we attempted to decrease heterogeneity and increase reliability of findings across studies by analysing and pooling study results according to common methodological criteria.

A random effects model of studies that compared interactive, tailored internet smoking cessation interventions with minimal control conditions (i.e. booklet/e-mail/static website/waiting list) found a significant treatment effect [RR 1.5; 95% confidence interval (CI) 1.2–1.9, P < 0.001]. However, there was significant heterogeneity in effect size. The increase in quit rates was larger when interactive, online smoking cessation interventions were compared with a booklet/e-mail than with a static website control condition (Fig. 1, subcategories 2 and 3).


Figure 1. Comparison with minimal, non-interactive, untailored condition. Forest plot; RR: rate ratio; 95% CI: 95% confidence interval

Download figure to PowerPoint

Separate analysis by control condition confirmed a significant non-heterogeneous treatment effect for comparisons with booklet or e-mail control condition (RR 1.8; 95% CI 1.4–2.3, P < 0.001, subcategory 2). However, the effect in studies with a static website control was non-significant and heterogeneous (subcategory 3), as this included a large trial [31] that did not detect a treatment effect. This study found differences in access to different websites used in the intervention condition; participants randomized to the most highly accessed intervention websites had significantly higher point-prevalence (but not continuous) abstinence rates than those randomized to the other websites. In a subanalysis that included only participants allocated to the heavily utilized intervention websites, we found that heterogeneity was no longer significant and results supported the treatment effect of tailored, interactive smoking cessation websites (RR 1.2, 95% CI 1.1–1.4, P < 0.001, subcategory 3a).

As about half of people who are continuously abstinent at 6 months achieve long-term smoking cessation, these data can be used to estimate permanent abstinence rates [45]. We therefore pooled the results of studies that provided half-year follow-up data on the percentage difference in success rates [24–26]. There was little heterogeneity; a random effects model suggests that interactive, online interventions increased abstinence by 17% (95% CI 12–21%), effectively doubling smoking cessation rates compared with minimal control conditions (RR 1.9, 95% CI 1.9–2.6, P < 0.001; Fig. 2). However, it should be noted that these data are based on point prevalence, not continuous abstinence rates, which is likely to result in greater relapse rates.


Figure 2. Comparison with minimal, non-interactive, untailored condition—6 months abstinence. Forest plot; RR: rate ratio; 95% CI: 95% confidence interval

Download figure to PowerPoint

Two studies compared an interactive internet intervention with either face-to-face counselling [30] or face-to-face motivational interviewing [32]. Neither study found evidence that the internet intervention increased smoking cessation rates compared with the control intervention (RR 0.6, 95% CI 0.3–1.1; Fig. 3, subcategory 1). Similarly, there was no evidence that web-based interventions increased abstinence in smokers also using face-to-face interventions (RR 1.3, 95% CI 0.8–2.1; Fig. 3; subcategory 2). However, the aggregate sample size in both these cases was small, so the power to detect an effect would be very limited.


Figure 3. Comparison with behavioural treatment. Forest plot; RR: rate ratio; 95% CI: 95% confidence interval

Download figure to PowerPoint

A further two studies evaluated the impact of ‘enhanced’ internet smoking cessation interventions on abstinence rates. One compared an interactive internet intervention that incorporated a mood management component with the same intervention lacking this component in two different settings: one was delivered in English and one in Spanish to appeal to Hispanic smokers in the United States [29]. Another study compared an interactive online intervention, including comprehensive, targeted components to increase smoking cessation, with a control that contained less explicit support for smoking cessation but aimed to increase physical activity as a way to stop smoking. Neither study found an effect of the enhanced intervention (RR 0.9, 95% CI 0.6–1.2; Fig. 4); in fact, when results were analysed separately according to language modality in the first trial, the addition of the mood management component depressed smoking cessation rates at 12 months for users of the English version of the intervention (χ2(1) = 4.4, P = 0.04).


Figure 4. Comparison with enhanced interactive internet intervention#. Forest plot; RR: rate ratio; 95% CI: 95% confidence interval. #Study results in Munoz et al.[29] are presented separately for the intervention of an English and Spanish websites but, for the purpose of this review, are collapsed across studies

Download figure to PowerPoint

As study methodologies were heterogeneous, we did not pool results according to follow-up point. However, individual studies that included several follow-ups provide some insight into the length of the treatment effect. Studies that did not find an initial treatment effect did not detect a late effect [27–30], whereas trials that reported a significant effect found this persisted throughout subsequent follow-ups [24–26,33], up to 12 months following some interventions.

Effect moderators and mediators

While a number of baseline variables associated commonly with smokers' chances of succeeding in quitting were identified (e.g. nicotine addiction [24], depression [29]), no interactions between intervention and baseline characteristics (including type of internet access—dial-up versus broadband) were observed when this was assessed.

Intention to stop smoking is an important putative moderator of intervention effectiveness. Pooled results from trials that did not require participants to intend to stop smoking in the near future (n = 4, Table 2) indicate no intervention effect (RR 1.2, 95% CI 0.8–1.9), but outcomes were heterogeneous (I2 = 77.6%; χ2(3) = 13.4, P = 0.004). In contrast, while equally heterogeneous (I2 = 70.3%; χ2(6) = 20.2, P = 0.003), the pooled results from trials recruiting smokers motivated to quit (n = 7) show a treatment effect (RR 1.3, 95% CI 1.0–1.7, P = 0.04).

As all interventions contained multiple components, it is important to try to determine which of these influenced outcomes. When pooling studies according to the length of treatment provided (see Table 3), there was no clear association with increased cessation rates. The pooled results of studies that were fully automated (n = 6, Table 2) showed a treatment effect (RR 1.4, 95% CI 1.0–2.0, P = 0.03), but results were heterogeneous (I2 = 73.8%; χ2(5) = 19.1, P = 0.002). In contrast, the pooled result of the five non-automated studies was not significant (RR 1.2, 95% CI 0.9–1.6), but also heterogeneous (I2 = 62.7%; χ2(4) = 10.7, P = 0.03).

Two studies that reported an increase in NRT use found that the effect of an interactive, online smoking cessation intervention was not mediated by usage rate of pharmacotherapy [25,33]. This was confirmed by two trials that found the same intervention to be effective with and without the provision of NRT [25,26]. These studies also observed that self-efficacy, at least partially, mediated the impact of the intervention. Lastly, in trials where this was assessed, a greater frequency of use of the provided internet intervention was associated with higher abstinence rates. However, it was not possible to evaluate further moderators and mediators, as interventions were both too diverse and did not provide sufficient detail to carry out further subanalyses.

Website utilization and satisfaction

As shown in Table 1, loss to follow-up and dropout from the web-based smoking cessation interventions was high, which is confirmed when looking at the usage of interactive websites. The proportion of users allocated to the treatment condition that logged on at least once was reasonably high (e.g. 70.2% [34]), but the use of the websites tapered quickly over time [27,28,30]. Brendryen et al.[25] contacted participants who had left the programme to identify reasons for their decision to discontinue and reported that this was due mainly to stressful life events or the intrusiveness of the web-based intervention.

To assess participants' views of the content of online interventions, Japunitch and colleagues [27] investigated the use of different components of their intervention and found that support tools (discussion groups and chatrooms, including an ‘ask the expert’ service) were the most popular module of their programme, while information tools (providing educational material about smoking cessation) were least popular. However, Swartz et al.[34] did not find any differences in the utilization of various modules of their online smoking cessation programme. Satisfaction with treatment programmes for smoking cessation was generally very high, at around 90% [25,26,33], and tailored interventions fared significantly better than non-tailored static internet interventions where this was compared [33].


  1. Top of page
  7. Declarations of interest
  8. Acknowledgements
  9. References

Any meta-analysis, such as this, that considers a very diverse and heterogeneous set of interventions will necessarily be limited in the generalizability of its conclusions. Moreover, given the lack of detailed description of interventions complicates further the analysis and evaluation of mediating and moderating effects. However, using stringent inclusion criteria and classifying interventions according to methodological similarities provides at least some measure of control and insight into the relative efficacy of the interventions. Studies evaluating tailored, interactive internet interventions compared with untailored written or web-based material detected a significant treatment effect across different follow-up times and treatment schedules; online interventions tested so far increased abstinence rates by an estimated 17% (95% CI 12–21%) at 6-month follow up. Given that unaided quit attempts result in abstinence rates of less than 5% at 6 months [46], internet interventions have the potential to more than double long-term abstinence rates. Moreover, the treatment effect of web-based smoking cessation interventions appears relatively stable over time, as differences in cessation rates between control and treatment groups did not dissipate at subsequent follow-ups.

These results compare favourably with findings from meta-analyses evaluating other smoking cessation interventions of similar potential reach which, typically, increase abstinence rates by between 1 and 3% [47–49]. However, while RCTs that compared internet interventions directly with self-help materials found a clear treatment effect, no direct comparisons with telephone counselling or brief advice were available. Abstinence rates from internet interventions were comparable to those obtained usually from more intensive, low-reach interventions such as face-to-face individual or group counselling [50,51]. Indeed, there were no differences in cessation rates in trials that compared internet-based interventions directly with face-to-face counselling, but these studies had small samples. Adding an online smoking cessation intervention to behavioural treatment or enhancing internet interventions did not confer any additional benefits. It therefore appears that online internet interventions for smoking cessation have the potential to combine the wide reach of less intensive interventions with the greater efficacy of face-to-face counselling. Findings from a study evaluating the efficacy of online interactive smoking cessation treatment for smokeless tobacco users indicate that such interventions can also be effective for other forms of tobacco use and different groups of smokers [52]. This is in agreement with the overall efficacy of interventions that were included in this review, targeting diverse populations including adolescents.

Similar to earlier reviews [15,21], we found considerable heterogeneity in the theoretical basis of interventions and no common rationale for the inclusion of various treatment components. As reported previously [6], tailoring of information was effective. Comparison with static websites suggests that an interactive user interface boosts the treatment effect, although it is unclear what form of interaction is required to confer this benefit. While it has been speculated that chat forums could aid smoking cessation through the provision of additional social support [53,54], it appears that completely automated interventions may be more effective. There was no evidence that increasing treatment complexity improved cessation rates, but this could reflect a ceiling effect, as all reviewed interventions were already highly complex. However, reducing the complexity of interventions can lead to a reduction in treatment efficacy [55], thus striking the right balance between user involvement and treatment intricacy is important for determining the impact of an intervention.

Few treatment moderators and mediators were examined explicitly in studies. While insufficient description precluded an in-depth analysis of moderator and mediator effects, stratifying studies according to common methodological characteristics allowed some delineation of possible moderating effects. There was generally no impact of socio-demographic characteristics, but motivation to quit appeared to influence the success of interventions. This lack of socio-demographic moderators may be surprising; however, given that all interventions attempted to match the provided treatment according to target population characteristics, one would not expect a differential outcome between various subgroups. This is in agreement with previous research that found few socio-demographic moderating variables and perceived relevance of the intervention as the only mediator of successful cessation [56].

Although there is considerable variability in the quality and usability of existing online interventions [13], use of, and reported satisfaction with, web-based smoking cessation interventions was generally high. None the less, usability testing by potential users of online programmes, rather than just by smoking cessation and web development experts, is likely to improve further existing online interventions [57].

Internet-based health interventions provide obvious methodological challenges [58,59] that need to be acknowledged. First, biochemical outcome validation is impractical in internet-based interventions as they span large geographic areas, and due to little contact between treatment provider and client are generally acknowledged not to be necessary [60]. Interestingly, studies that did validate abstinence did not report a difference in deception rate between intervention and control conditions [24,30]. Secondly, the use of the internet as a medium for delivering interventions may bias against people with low computer literacy, i.e. introduce a ‘digital divide’[58]. However, as the socio-demographic correlates of the digital divide have become blurred [61], web-based interventions have shown increasingly their potential to involve users excluded commonly from research [62].

Thirdly, the representativeness of trial participants needs to be established as substantial differences exist between smokers who are approached to participate and those who enrol with internet smoking cessation trials [63]. Only two studies in this review compared characteristics of study participants with those of smokers who were approached or with the general population [30,31]. Another issue is the ability of internet-based interventions to attract and retain participants in research trials [64], underlined by the fact that most of the included trials were underpowered due to a small sample and high attrition. This is likely to have undermined what can be said reliably about the magnitude of observed effects. Lastly, the relatively small number of included studies and variable methodological quality somewhat limits the generalizability of the reported findings. As with many behaviour change interventions [65], most studies provided inadequate detail on treatment characteristics and more needs to be done to improve reporting. In particular, most studies provided little information about structural factors that may be of importance to website users such as the navigational, organizational and graphic interface or pacing, the impact of which could therefore not be examined in this review.

With these caveats in mind, it appears that online interventions for smoking cessation are acceptable to users and of superior efficacy to other wide-reach interventions. Current, limited evidence also suggests that online interventions may have similar efficacy to face-to-face interventions. The impact of web-based smoking cessation interventions on quit rates seems to be relatively long-lasting and appears to be mediated by appropriate tailoring of information and personalizing the experience through interactive online components. Web-based internet interventions seem particularly effective for smokers intending to quit, but few other differentiating factors were observed. Some methodological concerns exist. It would be desirable for future trials of online smoking cessation interventions to provide adequate description, to evaluate the actual reach of interventions and the representativeness of included participants and to delineate clearly theoretical underpinnings of included components. More research is required to confirm the relative efficacy of interactive online interventions compared with static websites and face-to-face counselling.

Declarations of interest

  1. Top of page
  7. Declarations of interest
  8. Acknowledgements
  9. References

Lion Shahab has received an honorarium for a talk and travel expenses from a pharmaceutical company making smoking cessation products. Andy McEwen has received travel funding, honorariums and consultancy payments from manufacturers of smoking cessation products (Pfizer Ltd, Novartis and GSK Consumer Healthcare Ltd).


  1. Top of page
  7. Declarations of interest
  8. Acknowledgements
  9. References

This review was funded by the Department of Health, England. We would like to thank Robert West for help in preparing this manuscript.


  1. Top of page
  7. Declarations of interest
  8. Acknowledgements
  9. References
  • 1
    World Health Organization (WHO). The World Health Report 2002: Reducing Risks, Promoting Healthy Life. Geneva, Switzerland: WHO; 2002.
  • 2
    World Health Organization (WHO). Action for Tobacco Control. Guideline for Controlling and Monitoring the Tobacco Epidemic. Geneva, Switzerland: WHO; 1998.
  • 3
    US Department of Health and Human Services (USDHHS). Clinical Practical Guidelines. Treating Tobacco Use and Dependence—2008 Update. Rockville, MD: USDHHS, PHS; 2008.
  • 4
    Raw M., McNeill A. The prevention of tobacco-related disease. Addiction 1994; 89: 15059.
  • 5
    Abrams D. B., Orleans C. T., Niaura R. S., Goldstein M. G., Prochaska J. O., Velicer W. Integrating individual and public health perspectives for treatment of tobacco dependence under managed health care: a combined stepped-care and matching model. Ann Behav Med 1996; 18: 290304.
  • 6
    Noar S. M., Benac C. N., Harris M. S. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull 2007; 133: 67393.
  • 7
    Wang J., Etter J. F. Administering an effective health intervention for smoking cessation online: the international users of Stop-Tabac. Prev Med 2004; 39: 9628.
  • 8
    Cassell M. M., Jackson C., Cheuvront B. Health communication on the internet: an effective channel for health behavior change? J Health Commun 1998; 3: 719.
  • 9
    Murray E. Internet-delivered treatments for long-term conditions: strategies, efficiency and cost-effectiveness. Exp Rev Pharmacoeconom Outcome Res 2008; 8: 26172.
  • 10
    Modis T. The end of the internet rush. Technol Forecast Soc Change 2005; 72: 93843.
  • 11
    Pew Internet and American Life Project. Tracking Survey, November–December 2008. Available at: (accessed 20 May 2009).
  • 12
    Bock B., Graham A., Sciamanna C., Krishnamoorthy J., Whiteley J., Carmona-Barros R. et al. Smoking cessation treatment on the internet: content, quality, and usability. Nicotine Tob Res 2004; 6: 20719.
  • 13
    Etter J. F. A list of the most popular smoking cessation web sites and a comparison of their quality. Nicotine Tob Res 2006; 8: S27–S34.
  • 14
    Etter J. F. Internet-based smoking cessation programs. Int J Med Informat 2006; 75: 1106.
  • 15
    Kroeze W., Werkman A., Brug J. A systematic review of randomized trials on the effectiveness of computer-tailored education on physical activity and dietary behaviors. Ann Behav Med 2006; 31: 20523.
  • 16
    Van Den Berg M. H., Schoones J. W., Vliet Vlieland T. P. Internet-based physical activity interventions: a systematic review of the literature. J Med Internet Res 2007; 9: e26.
  • 17
    Blonde L., Parkin C. G. Internet resources to improve health care for patients with diabetes. Endocr Pract 2006; 12: 1317.
  • 18
    Vandelanotte C., Spathonis K. M., Eakin E. G., Owen N. Website-delivered physical activity interventions a review of the literature. Am J Prev Med 2007; 33: 5464.
  • 19
    Portnoy D. B., Scott-Sheldon L. A., Johnson B. T., Carey M. P. Computer-delivered interventions for health promotion and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988–2007. Prev Med 2008; 47: 316.
  • 20
    Strecher V. J. Computer-tailored smoking cessation materials: a review and discussion. Patient Educ Couns 1999; 36: 10717.
  • 21
    Walters S. T., Wright J. A., Shegog R. A review of computer and internet-based interventions for smoking behavior. Addict Behav 2006; 31: 26477.
  • 22
    Bessell T. L., McDonald S., Silagy C. A., Anderson J. N., Hiller J. E., SansomLN. Do internet interventions for consumers cause more harm than good? A systematic review. Health Expect 2002; 5: 2837.
  • 23
    Tramer M. R., Reynolds D. J., Moore R. A., McQuay H. J. Impact of covert duplicate publication on meta-analysis: a case study. BMJ 1997; 315: 63540.
  • 24
    An L. C., Klatt C., Perry C. L., Lein E. B., Hennrikus D. J., Pallonen U. E. et al. The RealU online cessation intervention for college smokers: a randomized controlled trial. Prev Med 2008; 47: 1949.
  • 25
    Brendryen H., KraftP. Happy Ending: a randomized controlled trial of a digital multi-media smoking cessation intervention. Addiction 2008; 103: 47884.
  • 26
    Brendryen H., Drozd F., KraftP. A digital smoking cessation program delivered through internet and cell phone without nicotine replacement (Happy Ending): randomized controlled trial. J Med Internet Res 2008; 10: e51.
  • 27
    Japuntich S. J., Zehner M. E., Smith S. S., Jorenby D. E., Valdez J. A., Fiore M. C. et al. Smoking cessation via the internet: a randomized clinical trial of an internet intervention as adjuvant treatment in a smoking cessation intervention. Nicotine Tob Res 2006; 8: S59–S67.
  • 28
    McKay H. G., Danaher B. G., Seeley J. R., Lichtenstein E., Comparing Gau J. Two web-based smoking cessation programs: randomized controlled trial. J Med Internet Res 2008; 10: e40.
  • 29
    Munoz R. F., Lenert L. L., Delucchi K., Stoddard J., Perez J. E., Penilla C. et al. Toward evidence-based internet interventions: a Spanish/English web site for international smoking cessation trials. Nicotine Tob Res 2006; 8: 7787.
  • 30
    Patten C. A., Croghan I. T., Meis T. M., Decker P. A., Pingree S., Colligan R. C. et al. Randomized clinical trial of an internet-based versus brief office intervention for adolescent smoking cessation. Patient Educ Couns 2006; 64: 24958.
  • 31
    Pike K. J., Rabius V., McAlister A., GeigerA. American Cancer Society's QuitLink: randomized trial of internet assistance. Nicotine Tob Res 2007; 9: 41520.
  • 32
    Prochaska J. O., Butterworth S., Redding C. A., Burden V., Perrin N., Leo M. et al. Initial efficacy of MI, TTM tailoring and HRIs with multiple behaviors for employee health promotion. Prev Med 2008; 46: 22631.
  • 33
    Strecher V. J., Shiffman S., West R. Randomized controlled trial of a web-based computer-tailored smoking cessation program as a supplement to nicotine patch therapy. Addiction 2005; 100: 6828.
  • 34
    Swartz L. H., Noell J. W., Schroeder S. W., Ary D. V. A randomised control study of a fully automated internet based smoking cessation programme. Tob Control 2006; 15: 712.
  • 35
    Peters J. L., Sutton A. J., Jones D. R., Abrams K. R., Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA 2006; 295: 67680.
  • 36
    L'Abbé K. A., Detsky A. S., O'Rourke K. Meta-analysis in clinical research. Ann Intern Med 1987; 107: 22433.
  • 37
    Review Manager (RevMan) [Computer Program], version 4.2 for Windows. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration; 2003.
  • 38
    Higgins J. P., Thompson S. G., Deeks J. J., Altman D. G. Measuring inconsistency in meta-analyses. BMJ 2003; 327: 55760.
  • 39
    Higgins J., Green S. Cochrane Handbook for Systematic Reviews of Interventions 4.2.5 [updated May 2005], 3rd edn. Chichester: John Wiley & Sons, Ltd; 2005.
  • 40
    An L. C., Perry C. L., Lein E. B., Klatt C., Farley D. M., Bliss R. L. et al. Strategies for increasing adherence to an online smoking cessation intervention for college students. Nicotine Tob Res 2006; 8: S7–12.
  • 41
    US Department of Health and Human Services (USDHHS). Treating Tobacco Use and Dependence. Clinical Practice Guideline, 00-0032 edn. Rockville, MD: USDHHS, PHS; 2000.
  • 42
    Bandura A. Social Foundations of Thought and Action. A Social Cognitive Theory. New York: Academic Press; 1986.
  • 43
    Pennebaker J. W. Writing about emotional experiences as a therapeutic process. Psychol Sci 1997; 8: 1626.
    Direct Link:
  • 44
    Prochaska J. O., DiClemente C. C. Stages and processes of self-change of smoking: toward an integrative model of change. J Consul Clin Psychol 1983; 51: 3905.
  • 45
    West R. The clinical significance of ‘small’ effects of smoking cessation treatments. Addiction 2007; 102: 5069.
  • 46
    Hughes J. R., Keely J., Naud S. Shape of the relapse curve and long-term abstinence among untreated smokers. Addiction 2004; 99: 2938.
  • 47
    Lancaster T., Stead L. Physician advice for smoking cessation. Cochrane Database Syst Rev 2004; 4: CD000165.
  • 48
    Stead L. F., Perera R., Lancaster T. Telephone counselling for smoking cessation. Cochrane Database Syst Rev 2006; 3: CD002850.
  • 49
    Lancaster T., Stead L. F. Self-help interventions for smoking cessation. Cochrane Database Syst Rev 2005; 3: CD001118.
  • 50
    Stead L. F., Lancaster T. Group behaviour therapy programmes for smoking cessation. Cochrane Database Syst Rev 2005; 2: CD001007.
  • 51
    Lancaster T., Stead L. F. Individual Behavioural Counselling for Smoking Cessation (Cochrane Review). The Cochrane Library, Issue 2. Chichester: John Wiley & Sons, Ltd; 2004.
  • 52
    Severson H. H., Gordon J. S., Danaher B. G., AkersL. evaluation of a web-based cessation program for smokeless tobacco users. Nicotine Tob Res 2008; 10: 38191.
  • 53
    Burri M., Baujard V., Etter J. F. A qualitative analysis of an internet discussion forum for recent ex-smokers. Nicotine Tob Res 2006; 8: S13–S19.
  • 54
    Cobb N. K., Graham A. L., Bock B. C., Papandonatos G., Abrams D. B. Initial evaluation of a real-world internet smoking cessation system. Nicotine Tob Res 2005; 7: 20716.
  • 55
    Etter J. F. Comparing the efficacy of two internet-based, computer-tailored smoking cessation programs: a randomized trial. J Med Internet Res 2005; 7: e2.
  • 56
    Strecher V. J., Shiffman S., West R. Moderators and mediators of a web-based computer-tailored smoking cessation program among nicotine patch users. Nicotine Tob Res 2006; 8: S95–101.
  • 57
    Stoddard J. L., Augustson E. M., Mabry P. L. The importance of usability testing in the development of an internet-based smoking cessation treatment resource. Nicotine Tob Res 2006; 8: S87–S93.
  • 58
    Glasgow R. E. eHealth evaluation and dissemination research. American J Preventive Med 2007; 32: S119–S126.
  • 59
    Strecher V. Internet methods for delivering behavioral and health-related interventions (eHealth). Annu Rev Clin Psychol 2007; 3: 5376.
  • 60
    Benowitz N. L., Jacob P., III, Ahijevych K., Jarvis M. J., Hall S., LeHouezec J. et al. Biochemical verification of tobacco use and cessation. Nicotine Tob Res 2002; 4: 14959.
  • 61
    Cotten S. R., Gupta S. S. Characteristics of online and offline health information seekers and factors that discriminate between them. Soc Sci Med 2004; 59: 1795806.
  • 62
    Danaher B. G., Hart L. G., McKay H. G., Severson H. H. Measuring participant rurality in web-based interventions. BMC Public Health 2007; 7: 228.
  • 63
    Stoddard J. L., Augustson E. M. Smokers who use internet and smokers who don't: data from the Health Information and National Trends Survey (HINTS). Nicotine Tob Res 2006; 8: S77–S85.
  • 64
    Koo M., Skinner H. Challenges of internet recruitment: a case study with disappointing results. J Med Internet Res 2005; 7: e6.
  • 65
    Michie S., Abraham C. Advancing the science of behaviour change: a plea for scientific reporting. Addiction 2008; 103: 140910.