Criteria for considering studies for this review
Types of studies
Randomised controlled trials, cluster randomised controlled trials with at least two intervention sites and two comparator sites, quasi-randomised trials and interrupted time series studies (ITS) with a clearly defined point in time when the intervention occurred and at least three data points before and after the intervention.
Types of participants
People who smoke and are receiving care in a healthcare delivery setting.
Types of interventions
System change interventions for smoking cessation are policies and practices designed by organisations to integrate the identification of smokers and the subsequent offering and receipt of evidence-based tobacco dependence treatments into the usual care (Fiore 2007). Thus interventions which have been developed for identifying people who smoke, documenting smoking status and providing tobacco dependence treatment at different healthcare settings (primary, secondary or tertiary care settings) will be included in the review.
Studies utilising the components of Fiore et al’s model will be considered (Fiore 2007).
Implement a system for identifying smokers and documenting the tobacco use status in every clinic and hospital;
Provide education, resources and feedback to promote provider intervention;
Dedicating staff to provide smoking cessation treatment and assess its delivery in staff performance evaluations;
Promote hospital policies that support and provide smoking cessation services;
Include tobacco dependence treatments (both counselling and pharmacotherapy) identified as effective; and
Reimburse providers for the delivery of effective tobacco dependence treatments and include these services among the defined duties of them.
Those studies focusing only on training health professionals or identification of smokers (electronic health records) or smoking cessation counselling without a system change approach will not be considered. It should be designed for integrating the provision of smoking cessation services within the routine delivery of health care. Each potential study will be reviewed by two authors, including a content expert before including in the review.
Types of outcome measures
Following the standard methodology of the Cochrane Tobacco Addiction Group, the primary outcome will be abstinence from smoking at the longest follow-up, assessed as point prevalence (defined as prevalence of abstinence during a time window immediately preceding the follow-up) and/or continued or prolonged abstinence (defined as abstinence between quit day or predetermined grace period and a follow-up time). The strictest available criteria to define abstinence will be used; thus continuous or prolonged abstinence will be preferred over point prevalence, and biochemically validated abstinence over self reported abstinence. We will distinguish between short term abstinence, assessed less than six months from the initiation of intervention with a patient, and long term abstinence, assessed after six months or longer.
Studies that do not assess smoking cessation will be eligible for inclusion if they report any secondary outcome and meet other inclusion criteria. The classification of primary and secondary outcomes in each included study will be examined and reported.
Increase in the provision of smoking cessation support services as a part of routine care measured as either organisational or patient or health professional level strategies.
Organisational level outcome measures may include number of smokers identified and smoking status documented, and number of health professionals trained or dedicated to provide cessation support.
Patient level outcome measures may include number of smokers who were counselled, given self help materials, offered nicotine replacement therapy (NRT) or other pharmacotherapy, nominated a quit date and given follow-up appointment.
Health professional level outcome measures may include number of referrals made to other health professionals and/or to local smoking cessation services.
We will present the main outcomes of the review in a Summary of Findings table.
Search methods for identification of studies
We will search the following databases:
Cochrane Central Register of Controlled Trials (CENTRAL);
MEDLINE (1946 to present);
EMBASE (1947 to present);
PsycINFO (1806 to present); and
CINAHL (1938 to present).
The search strategies will be developed to comprise searches both for key words and Medical Subject Headings/Emtree. We will aim to identify articles reporting randmised control trials (RCTs), cluster RCTs, Quasi RCTs and ITS studies that comprise intervention and a measure of the effect on system change approach. No language restriction will be employed. The strategy for MEDLINE is presented in Appendix 1.
Searching other resources
Studies will also be identified by screening references given in relevant reviews and identified studies (citation tracking). Personal bibliographies and communication with experts in the field will also be considered to identify any hidden studies.
Data collection and analysis
Selection of studies
DT will implement the search strategy and the search results will be merged using reference management software (EndNote®). The titles and abstracts of the studies will be reviewed for possible inclusion, and those selected will be subjected for full text assessment. Multiple reports of the same study will be linked together. Two authors (DT and JG) will independently assess all the full text articles retrieved, and those studies meeting the inclusion criteria will be included in the review. Any discrepancies will be resolved by discussing with the third author who will act as an arbiter. A content area expert (BB) will act as an arbiter for disagreement about the intervention or content of the study. Methodological discrepancies will be checked by another arbiter (MA) who is an expert in clinical trials and meta-analysis. Characteristics of the studies excluded, (after full text assessment) including the reason for exclusion, will be listed and reported.
Data extraction and management
Two authors (DT and JG) will extract data independently and categorise trials for subgroup analysis. A pre-tested (pilot tested), standardised data collection form will be employed. Data from the data collection forms will be entered into RevMan 5.2 for analysis. Authors of the studies where data are not available or unclear will be contacted by e-mail. The following information will be extracted from each of the selected studies:
lead and corresponding authors’ information;
date of publication;
location and setting;
methods of recruitment and inclusion criteria;
methods of randomisation, allocation, concealment and blinding;
study design, duration and follow-up details;
characteristics of participants (e.g. age, sex and smoking status);
specific details of the intervention (type, duration, content, format and delivery of intervention, use of pharmacotherapy, adherence to therapy and information about the providers);
control group component;
number of participants in each arm;
outcome measures and definitions including any biochemical validation, and time point at which they are measured and reported;
results: estimate of effect with confidence intervals and subgroup analysis (summary data of intervention and control group will be entered separately into RevMan, where effect estimates can be calculated) and missing data;
funding, and declaration of interest for the primary investigators;
conclusion of the authors; and
additional comments and information.
If studies are reported in more than one publication (e.g. different time points of the study) the data from all publications will be extracted in separate data collection forms and combined. If there is one full journal article and multiple conference abstracts are available, only the journal article will be considered. Any disagreement in the data collection process will be resolved by discussing with a third author (MA).
Assessment of risk of bias in included studies
Two review authors (DT and JG) will independently assess the risk of bias of included studies, with any disagreements resolved by discussion and consensus, and by consulting a third review author, where necessary.
The following criteria for assessing risk of bias will be implemented:
intervention independent of other changes;
pre-specified effect shape;
intervention unlikely to affect data collection;
incomplete outcome data;
selective outcome reporting; and
Each criterion will be judged on a 3-point scale for bias ‘low risk’, ‘high risk’ and ‘unclear risk’ (Higgins 2011) and a risk of bias table will be constructed.
‘Low risk’ when there is a low risk of bias across all key domains.
‘Unclear risk’ when there is an unclear risk of bias in one or more of the key domains.
‘High risk’ when there is a high risk of bias in one or more of the key domains.
For each included study, a summary assessment of risk of bias will be provided.
Measures of treatment effect
Wherever possible, a risk ratio (quitters in treatment group/total randomised to the treatment group)/(quitters in control group/total randomised to the control group) will be provided for the outcome of each trial.
Unit of analysis issues
In the case of trials with repeated observations, the longest follow-up will be considered for the analysis (Higgins 2011).
In the case of cluster RCTs, an adjusted estimate of the required effect measure will be extracted from an analysis that properly accounts for the cluster design. Where such data are unavailable, an approximate analysis will be performed if the required information can be obtained (Higgins 2011). If a comparison is re-analysed, the P value will be annotated with a comment ‘re-analysed’.
In the case of trials with multiple arms, only arms that meet the eligibility criteria will be included in the review. If there are more than one eligible intervention arms, all the relevant experimental groups will be combined to create a single pair-wise comparison to avoid the problem of including same group of participants twice in the same meta-analysis. If multiple intervention arms are eligible and not comparable, each pair-wise comparison will be included separately, but with shared intervention arms divided out approximately evenly among the comparisons (Higgins 2011).
Dealing with missing data
The number of participants lost to follow-up will be reported by group, where available. If required, we will contact the study authors for more information. For quit rates, an intention to treat analysis will be followed. This assumes that people lost to follow-up continued smoking and will be included in the denominator for calculating relative risk.
Assessment of heterogeneity
Heterogeneity will be explored visually using tables and forest plots comparing effect sizes of studies grouped according to potential effect modifiers. This will include:
type of intervention (e.g. identification of smokers, documentation of smoking status, treatment, training of health professionals, feedback of services etc.);
intensity of intervention (e.g. counselling, pharmacotherapy, both counselling and pharmacotherapy, number of follow-ups etc.);
type of health professional involved;
settings (primary, secondary and tertiary);
study design (RCTs, cluster RCTs, quasi-RCTs and ITS studies); and
quality of studies.
If sufficient number of homogenous studies are available, statistical heterogeneity between study results will be assessed using Chi2 test for homogeneity (with significance defined at the alpha-level of 10%) and any statistical heterogeneity will be quantified using I² statistic. Pooling of data using a meta-analysis will be considered if the heterogeneity is less than 50% (Higgins 2011).
Assessment of reporting biases
The publication bias will be assessed using funnel plots if there are sufficient number of studies.
A narrative synthesis of the included studies will be presented. The major characteristics and results will be reported. We will group the studies under different definitions of system change. If studies or groups of studies are sufficiently similar in terms of participants, intervention, outcome and/or methodology, we will consider pooling the data statistically. If meta-analysis is appropriate, a random effects model will be used as we suspect clinical and/or methodological heterogeneity between studies sufficient to suggest that intervention effects may differ between trials. If there is a substantial heterogeneity and formal meta-analysis techniques are not possible, the median (IQR - interquartile range) effect size will be calculated to quantify the expected magnitude of improvement.
Subgroup analysis and investigation of heterogeneity
We will categorise trials according to different system change interventions and type of studies identified. We may consider the following categories based on the nature of identified studies:
studies in different health care settings (primary, secondary and tertiary);
studies which followed ‘Russell Standards’ – a common standard for reporting outcome criteria in smoking cessation studies. ‘Russell Standards’ include six criteria describing abstinence, duration of abstinence, biochemical verification, intention to treat analysis, protocol violations and blinding in smoking cessation studies (West 2005);
randomised controlled studies and non-randomised studies;
studies using self report alone and biochemically verified abstinence; and
studies with minimal (less than three components) and intensive system change intervention (three or more components).
If there is an appropriate number of studies, we will consider the pooling of data and conducting analyses to determine any differential effect between different types of interventions.
If there are sufficient data to conduct a meta-analysis, we will consider whether the results are sensitive to the exclusion of trials judged to have a high risk of bias. We will also consider doing sensitivity analysis if there are issues identified during the review. For example, heterogeneity due to the presence of one or two outlying studies. In such cases we will consider doing analyses both including and excluding these studies. If conducted, the results of the sensitivity analyses will be reported in summary tables.