Strategies to improve retention in randomised trials

  • Review
  • Methodology

Authors


Abstract

Background

Loss to follow-up from randomised trials can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to reduce loss to follow-up and improve retention but few have been formally evaluated.

Objectives

To quantify the effect of strategies to improve retention on the proportion of participants retained in randomised trials and to investigate if the effect varied by trial strategy and trial setting.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, PreMEDLINE, EMBASE, PsycINFO, DARE, CINAHL, Campbell Collaboration’s Social, Psychological, Educational and Criminological Trials Register, and ERIC. We handsearched conference proceedings and publication reference lists for eligible retention trials. We also surveyed all UK Clinical Trials Units to identify further studies.

Selection criteria

We included eligible retention trials of randomised or quasi-randomised evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

Data collection and analysis

We contacted authors to supplement or confirm data that we had extracted. For retention trials, we recorded data on the method of randomisation, type of strategy evaluated, comparator, primary outcome, planned sample size, numbers randomised and numbers retained. We used risk ratios (RR) to evaluate the effectiveness of the addition of strategies to improve retention. We assessed heterogeneity between trials using the Chi2 and I2 statistics. For main trials that hosted retention trials, we extracted data on disease area, intervention, population, healthcare setting, sequence generation and allocation concealment.

Main results

We identified 38 eligible retention trials. Included trials evaluated six broad types of strategies to improve retention. These were incentives, communication strategies, new questionnaire format, participant case management, behavioural and methodological interventions. For 34 of the included trials, retention was response to postal and electronic questionnaires with or without medical test kits. For four trials, retention was the number of participants remaining in the trial. Included trials were conducted across a spectrum of disease areas, countries, healthcare and community settings. Strategies that improved trial retention were addition of monetary incentives compared with no incentive for return of trial-related postal questionnaires (RR 1.18; 95% CI 1.09 to 1.28, P value < 0.0001), addition of an offer of monetary incentive compared with no offer for return of electronic questionnaires (RR 1.25; 95% CI 1.14 to 1.38, P value < 0.00001) and an offer of a GBP20 voucher compared with GBP10 for return of postal questionnaires and biomedical test kits (RR 1.12; 95% CI 1.04 to 1.22, P value < 0.005). The evidence that shorter questionnaires are better than longer questionnaires was unclear (RR 1.04; 95% CI 1.00 to 1.08, P value = 0.07) and the evidence for questionnaires relevant to the disease/condition was also unclear (RR 1.07; 95% CI 1.01 to 1.14). Although each was based on the results of a single trial, recorded delivery of questionnaires seemed to be more effective than telephone reminders (RR 2.08; 95% CI 1.11 to 3.87, P value = 0.02) and a 'package' of postal communication strategies with reminder letters appeared to be better than standard procedures (RR 1.43; 95% CI 1.22 to 1.67, P value < 0.0001). An open trial design also appeared more effective than a blind trial design for return of questionnaires in one fracture prevention trial (RR 1.37; 95% CI 1.16 to 1.63, P value = 0.0003).

There was no good evidence that the addition of a non-monetary incentive, an offer of a non-monetary incentive, 'enhanced' letters, letters delivered by priority post, additional reminders, or questionnaire question order either increased or decreased trial questionnaire response/retention. There was also no evidence that a telephone survey was either more or less effective than a monetary incentive and a questionnaire. As our analyses are based on single trials, the effect on questionnaire response of using offers of charity donations, sending reminders to trial sites and when a questionnaire is sent, may need further evaluation. Case management and behavioural strategies used for trial retention may also warrant further evaluation.

Authors' conclusions

Most of the retention trials that we identified evaluated questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow-up. Monetary incentives and offers of monetary incentives increased postal and electronic questionnaire response. Some other strategies evaluated in single trials looked promising but need further evaluation. Application of the findings of this review would depend on trial setting, population, disease area, data collection and follow-up procedures.

Plain language summary

Methods that might help to keep people in randomised trials

Background

Most trials follow people up to collect data through personal contact after they have been recruited. Some trials get data from other sources, such as routine collected data or disease registers. There are many ways to collect data from people in trials, and these include using letters, the internet, telephone calls, text messaging, face-to-face meetings or the return of medical test kits. Most trials have missing data, for example, because people are too busy to reply, are unable to attend a clinic, have moved or no longer want to participate. Sometimes data has not been recorded at study sites, or are not sent to the trial co-ordinating centre. Researchers call this 'loss to follow-up', 'drop out' or 'attrition' and it can affect the trial's results. For example, if the people with the most or least severe symptoms do not return questionnaires or attend a follow-up visit, this will bias the findings of the trial. Many methods are used by researchers to keep people in trials. These encourage people to send back data by questionnaire, return to a clinic or hospital for trial-related tests, or be seen by a health or community care worker.

Study characteristics

This review identified methods that encouraged people to stay in trials. We searched scientific databases for randomised studies (where people are allocated to one of two or more possible treatments in a random manner) or quasi-randomised studies (where allocation is not really random, e.g. based on date of birth, order in which they attended clinic) that compared methods of increasing retention in trials. We included trials of participants from any age, gender, ethnic, cultural, language and geographic groups.

Key results

The methods that appeared to work were offering or giving a small amount of money for return of a completed questionnaire and enclosing a small amount of money with a questionnaire with the promise of a further small amount of money for return of a filled in questionnaire. The effect of other ways to keep people in trials is still not clear and more research is needed to see if these really do work. Such methods are shorter questionnaires, sending questionnaires by recorded delivery, using a trial design where people know which treatment they will receive, sending specially designed letters with a reply self addressed stamped envelope followed by a number of reminders, offering a donation to charity or entry into a prize draw, sending a reminder to the study site about participants to follow-up, sending questionnaires close to the time the patient was last followed-up, managing peoples' follow-up, conducting follow-up by telephone and changing the order of questionnaire questions.

Quality of evidence

The methods that we identified were tested in trials run in many different disease areas and settings and, in some cases, were tested in only one trial. Therefore, more studies are needed to help decide whether our findings could be used in other research fields.

எளியமொழிச் சுருக்கம்

சீரற்ற சோதனைகளில் மக்களை தக்க வைக்க உதவும் வழிமுறைகள்

பின்புலம்

பெரும்பாலான சோதனைகள், மக்களை ஆய்வில் சேர்த்த பின்னர், தனிப்பட்ட தொடர்பின் மூலம் தரவை சேகரிக்க அவர்களை பின் தொடரும். சில சோதனைகள், வழக்கமான சேகரிக்கப்பட்ட தரவு அல்லது நோய் பதிவேடுகள் போன்ற மற்ற ஆதாரங்களில் இருந்து தரவை கிடைக்கப் பெறும். பரிசோதனைகளில், மக்களிடமிருந்து தரவுகளை சேகரிக்க பல வழிகள் உள்ளன, மற்றும் இவை, கடிதங்கள், இணையம், தொலைபேசி அழைப்புகள், குறுஞ்செய்தி அனுப்புதல், முக-முகமான சந்திப்புகள், அல்லது மருத்துவ சோதனை கருவிகளை திரும்ப பெறுவது போன்றவற்றுள் அடங்கும். பதிலளிக்க முடியாமல் மக்கள் மிகவும் நேரமின்மையோடு இருப்பது, ஒரு மருத்துவ சிகிச்சையகத்திற்கு செல்ல முடியாமல் இருப்பது, இடம் மாறி செல்வது அல்லது இனிமேல் பங்கேற்க வேண்டாம் என்று முடிவெடுப்பது போன்ற காரணங்களால், பெரும்பாலான சோதனைகள் தொலைந்த தரவை கொண்டுள்ளன. சில நேரங்களில், ஆய்வு தளங்களில் தரவு பதிவு செய்யப்பட்டிருக்காது, அல்லது சோதனை ஒருங்கிணைப்பு மையத்திற்கு அனுப்பி வைக்கப்பட்டிருக்காது. இதை, ஆராய்ச்சியாளர்கள் 'பின்-தொடர் இழப்பு', 'பங்கேற்பாளர் வெளியேறுவது' அல்லது 'தேய்வு' என்று அழைப்பர், மற்றும் இது சோதனை முடிவுகளை பாதிக்கக் கூடும். உதாரணமாக, மிக அதிகமான அல்லது குறைந்த தீவிர அறிகுறிகள் உள்ளவர்கள், வினாப்பட்டியல்களை திரும்பி தராமல் போனால் அல்லது ஒரு பின்தொடர்ந்த சிகிச்சை அமர்விற்கு வராமல் போனாலோ, அவை சோதனை கண்டுபிடிப்புகளில் பரபட்சத்தை ஏற்படுத்தும். சோதனைகளில் மக்களைத் தக்க வைக்க, ஆராய்ச்சியாளர்கள் பல வழிமுறைகளை பயன்படுத்துகின்றனர். கேள்வித்தாள் மூலம் தரவை திரும்ப அனுப்ப, சோதனை-தொடர்பான ஆய்வுகளுக்கு ஒரு சிகிச்சை மையத்திற்கோ அல்லது மருத்துவமனைக்கோ திரும்ப வர, அல்லது ஒரு ஆரோக்கிய அல்லது சமூக பராமரிப்பு பணியாளர் மூலம் பார்க்கப்பட இவை மக்களுக்கு ஊக்கமளிக்கும்.

ஆய்வு பண்புகள்

சோதனைகளில் மக்களைத் தக்க வைக்க உற்சாகப்படுத்துகிற வழிமுறைகளை இந்த திறனாய்வு கண்டறிந்தது. சோதனைகளில் மக்களைத் தக்க வைத்தலை அதிகரிக்கும் வழிமுறைகளை ஒப்பிட்ட சீரற்ற சமவாய்ப்பு ஆய்வுகள் (இரண்டில் ஒன்று அல்லது அதற்கும் மேல் வாய்ப்புள்ள சிகிச்சைகளுக்கு மக்களை சீரற்ற முறையில் ஒதுக்குதல்) அல்லது அரை-சீரற்ற சமவாய்ப்பு ஆய்வுகளை (உண்மையான சீரற்ற முறையில் மக்களை ஒதுக்காது, எ.கா. பிறந்த நாள், சிகிச்சை மையத்தில் கலந்து கொண்ட ஒழுங்கை அடிப்படையாக கொண்டு ஒதுக்குதல்) அறிவியல் தரவுத்தளங்களில் நாங்கள் தேடினோம். அனைத்து வயது, பாலினம், இனம் , கலாச்சாரம், மொழி, புவியியல் குழுக்களிலிருந்த பங்கேற்பாளர்களை கொண்ட சோதனைகளை நாங்கள் சேர்த்துள்ளோம்.

முக்கிய கண்டுப்பிடிப்புகள்

ஒரு நிறைவு செய்த கருத்தறியும் வினாப்பட்டியலை திருப்பி தரப்படும் போது சிறிய அளவிலான பணம் அளிக்கப்படுவது, மற்றும் நிறைவு செய்யப்படும் வினாப்பட்டியலை திருப்பி தருவதற்கு மேலும் ஒரு சிறிய அளவிலான பணம் அளிக்கப்படும் என்ற வாக்குறுதியுடன் வினாப்பட்டியலோடு இணைக்கப்பட்ட ஒரு சிறிய அளவிலான பணம் ஆகிய வழிமுறைகள் பயனளித்ததாக தெரிகிறது. சோதனைகளில் மக்களை தக்க வைப்பதற்கு மற்ற வழிகளின் விளைவு பற்றி இன்னும் தெளிவாகவில்லை, மேலும் இவை உண்மையில் பயனளிக்குமா என்பதை அறிய அதிக ஆராய்ச்சி தேவைப்படுகிறது. குறுகிய வினாப்பட்டியல்கள், பதிவு தபால் முறை மூலம் வினாப்பட்டியல்கள் அனுப்புதல், பங்கேற்பாளர்கள் தாங்கள் எந்த சிகிச்சை பெறுகிறோம் என்று தெரிந்த சோதனை வடிவம், சுய-விலாசமிட்ட உறையுடன் விசேஷமாக வடிவமைக்கப்பட்ட திருப்பி அனுப்பப்படும் தபால்கள் மற்றும் அதனை தொடர்ந்த நினைவூட்டல்கள், தரும நன்கொடை வழங்குதல் அல்லது பரிசு குலுக்கலில் நுழைதல், பங்கேற்பாளர் பின்-தொடர்தல் குறித்து ஆய்வு மையத்திற்கு நினைவூட்டல் அனுப்புதல், கடைசியாக நோயாளி பின்-தொடரப்பட்ட சமயத்திற்கு ஒட்டியே வினாப்பட்டியல்களை அனுப்புதல், பின்-தொடர்தல் மேலாண்மை, தொலைபேசி மூலம் பின்-தொடர்தல் நடத்தி மற்றும் வினாப்பட்டியலின் கேள்விகளை ஒழுங்கை மாற்றுதல் ஆகியவை அத்தகைய வழிமுறைகளில் அடங்கும்.

சான்றின் தரம்

நாங்கள் அடையாளம் காட்டிய முறைகள் பல்வேறு நோய் பகுதிகள் மற்றும் அமைப்புகளில் நடத்தப்பட்ட சோதனைகளில் ஆய்வு செய்யபட்டதாகும், மற்றும், சில சந்தர்ப்பங்களில், ஒரே ஒரு சோதனையில் பரிசோதிக்கப் பட்டதாகும். எனவே, பிற ஆராய்ச்சி துறைகளில் நமது கண்டுப்பிடிப்புகளை பயன்படுத்த முடியும் என்பதை தீர்மானிக்க உதவ அதிக ஆய்வுகள் தேவைப்படுகிறது.

மொழிபெயர்ப்பு குறிப்புகள்

மொழி பெயர்ப்பாளர்கள்: சிந்தியா ஸ்வர்ணலதா ஸ்ரீகேசவன், தங்கமணி ராமலிங்கம், ப்ளசிங்டா விஜய், ஸ்ரீகேசவன் சபாபதி.

Background

Description of the problem or issue

Randomised trials are the gold standard for evaluating the effectiveness and efficacy of interventions. Non-response or loss to follow-up within study groups in randomised trials can compromise study findings by reducing the power of a study to detect a true difference between the control and the intervention group. Differential loss to follow-up may lead to bias through exaggerated effects in favour of one of the groups. This can affect the generalisability and internal validity of the trial and the results (Fewtrell 2008; Schulz 2002).

Missing data from loss to follow-up can be dealt with statistically by various methods including, for example, imputing values based on valid assumptions about the missing data to give a conservative estimate of the treatment effect. However, the risk of bias still remains when trials do not collect adequate data to give accurate estimates (Hollis 1999). Schulz and colleagues suggested that less than 5% loss to follow-up may lead to minimum bias, while 20% loss to follow-up can threaten trial validity, although the pattern of loss to follow-up by treatment may also be an important factor (Schulz 2002). Loss to follow-up from randomised trials can sometimes go unreported and using different, but plausible, assumptions about outcomes for participants lost to follow-up can change the results of randomised trials.

A number of trials have retrospectively examined the predictors of loss to follow-up in different disease areas (Arnow 2007; Snow 2007; Villarruel 2006). In a trial for the treatment of chronic major depression, Arnow examined the predictors of time to, and reason for, dropout of participants (Arnow 2007). Ethnic minorities and participants with comorbid anxiety were more likely to drop out. In a randomised trial of a human immunodeficiency virus (HIV) prevention intervention for Latino youths, English speakers were more likely to attend follow-up (Villarruel 2006). Snow examined the predictors of clinic attendance and dropout at the 11-year follow-up of the Lung Health study (Snow 2007). Age, gender, number of cigarettes smoked per day, marital status and whether participant's children smoked were predictors of clinic attendance. These analyses showed that attendance for follow-up can be trial and disease specific. An awareness of these factors can help trialists decide which strategies to adopt to improve retention in their randomised trial.

Description of the methods being investigated

Strategies to improve trial retention include those designed to generate maximum data return or compliance to follow-up procedures. These can include frequency and timing of follow-up visits (follow-up shortly after randomisation versus long-term follow-up), nature of the outcome to be measured (survey based self reported outcomes versus morbidity or mortality reporting), target of the intervention (participants versus providers versus trial sites), and type of intervention (incentives versus communication strategies versus participant case management).

How these methods might work

These retention strategies are designed to motivate participants (Leathem 2009), or the trial site to continue participating in a trial once they have been recruited and randomised. Some strategies are designed to encourage participants to identify with the trial and to promote a sense of value and belonging, for example, using trial identity cards. Other strategies are designed to keep participants engaged in the trial, for example, by sending participant newsletters. To encourage a proactive approach to trial retention, strategies can be designed to target participants directly through letters, emails, telephone calls or to target them via the clinicians involved in participant follow-up, for example, through regular communication with trial sites. Strategies have been specifically developed to promote retention in areas of research where it is particularly challenging, such as mental health (Furimsky 2008; Loue 2008), weight loss ( Couper 2007; Goldberg 2005), rare diseases (McKinstry 2007), substance abuse (El Khorazaty 2007), research involving minority ethnic groups (Eakin 2007; Loftin 2005; Villacorta 2007), and vulnerable groups such as older people (Burns 2008) or people with HIV (Anastasi 2005).

Why it is important to do this review

As drop-out or incomplete data causes problems in the conduct, analysis and interpretation of randomised trials, it is important to identify retention strategies that minimise this loss as far as possible.

Davis and colleagues conducted a review of community-based trials published from 1990 to 1999 and described retention strategies and retention outcomes for this area (Davis 2002). Robinson and colleagues conducted a systematic review of strategies for retaining study participants (Robinson 2007). While both reviews identified studies providing data on retention rates from primary studies and strategies used to promote retention, these were not evaluated quantitatively in either review.

A systematic review of strategies to retain participants in population-based cohort studies found that providing incentives was consistently associated with retention in these studies and that response generally increased with increasing incentive value (Booker 2011). Reminder letters, repeat questionnaires and reminder calls also increased response rates. Furthermore, the Edwards et al. Cochrane methodology review on methods to increase response rates to postal and electronic questionnaires found that including monetary incentives, keeping the questionnaire short and contacting people before sending the questionnaire were ways to increase response rates (Edwards 2009). That review was not restricted to research exclusively within randomised trials and covered both healthcare and non-healthcare settings and it is difficult to know which of these strategies would be applicable to randomised trials in health care. Reasons for drop-out in cohort studies and surveys may differ from those in randomised trials. For example, in trials, participants may be randomised to a study group that is not their preferred choice and factors around randomisation and the type of intervention mean that strategies increasing retention in cohort studies and surveys cannot necessarily be extrapolated to randomised trials.

The challenges of boosting recruitment to randomised trials is often described alongside retention in the literature. Some similar strategies may be used in an attempt to both increase recruitment and improve retention, such as giving incentives together with extra information. Rendell et al. assessed the evidence for the effect of disincentives and incentives on the extent to which clinicians invite eligible people to participate in randomised trials of healthcare interventions (Rendell 2007). No randomised trials of interventions were identified and the authors concluded that some aspects of the conduct of the trial might affect a clinician's willingness to invite people to participate, for example, the way the clinician is invited to take part and the availability of support staff. In another Cochrane methodology review, Treweek et al. assessed strategies to improve recruitment to research studies (Treweek 2010), but recruitment to trials presents different challenges to participant engagement and follow-up. For example, strategies to market a trial and win over participants during the recruitment phase may be different to strategies to keep participants engaged in a trial (Francis 2007).

Many untested strategies are used by researchers to try to improve retention in randomised trials. Therefore, because loss to follow-up can compromise the validity of a trial's findings, delay results and, in some circumstances, increase the costs of the research, a systematic review is needed to assess the effect of strategies to improve retention in randomised trials.

Objectives

To quantify the effect of strategies to improve retention in randomised trials.

To investigate if the effect varies by the type of strategy, trial setting and healthcare area.

Methods

Criteria for considering studies for this review

Types of studies

We included completed randomised trials that compared strategies to increase retention embedded in host randomised trials (hereafter referred to as retention trials). The retention trials were embedded in real trials (host trials) and not hypothetical trials. The retention trials included at least one randomised comparison of two or more strategies to improve retention, or compared one or more strategies with no strategy. In anticipation of few trials, we included retention trials if they were randomised or quasi-randomised (e.g. had used alternation, date of birth or case record number as a method of allocating participants) (Lefebvre 2008).

Strategies to improve retention were designed for impact after participants were recruited and randomised to either the intervention or control group of the main and the retention trial. We included trials to increase response to postal and electronic questionnaire. We excluded trials to increase recruitment only. We excluded cohort studies with embedded randomised retention trials, which were the subject of a separate systematic review (Booker 2011).

Types of data

We included randomised and quasi-randomised retention trials within the context of a host randomised trial with participants from any age, gender, ethnic, cultural, language and geographic groups. We included unpublished and published participant retention data from randomised trials addressing healthcare (including all disciplines and disease areas) and non-healthcare (education, social sciences) topics. We also included trials set in the community that were healthcare related.

Types of methods

We considered any strategy aimed at increasing retention, directed towards the clinician, researcher or participant. We included strategies compared with each other or with usual study procedures. We also included trials with any combination of strategies to increase retention. Strategies could be participant or trial management focused and include any of the following:

  • strategies to motivate participants and clinicians (e.g. incentives or gifts);

  • strategies to improve communication with participants or trial sites (e.g. enhanced letters);

  • methodology strategies (e.g. shorter length of follow-up or variation in follow-up visit frequency);

  • strategies to improve social support for participant retention.

Types of outcome measures

Primary outcomes

We used retention (the proportion of participants retained) at the primary analysis point as defined in each individual retention trial as the primary outcome because it is easier to interpret than attrition/loss to follow-up (i.e. the proportion lost or not retained). In cases where the time point for measurement of the primary outcome was not predefined, we took the first time point reported for analysis. In most cases, this was final response. If retention at a number of time points was reported and no clear time point for the primary outcome for the retention trial was stated, we took data for the nearest time point to the intervention in the retention trial analyses.

Secondary outcomes

Retention of participants at secondary analysis points.

Search methods for identification of studies

We designed a search strategy to identify published and unpublished randomised and quasi-randomised trials that assessed strategies to improve retention in randomised trials in healthcare, education and social science settings. We searched bibliographic databases for published trials and trial registers for trials that had not been fully published, or were unpublished or ongoing. We applied no language restrictions.

Electronic searches

Each search comprised an established filter to identify randomised trials plus free-text terms and database subject headings relating to reducing loss to follow-up or increasing retention (Appendix 1). Electronic databases searched included:

  • the Cochrane Central Register of Controlled Trials (CENTRAL) (to May 2012);

  • PreMEDLINE (to April 2010);

  • MEDLINE (1950 to May 2012) (Appendix 2), EMBASE (1980 to May 2012) (Appendix 3) and PsycINFO (1806 to May 2012) (Appendix 4), searched using an Ovid platform;

  • Database of Abstracts of Reviews of Effects (DARE, in The Cochrane Library May 2012);

  • CINAHL (Cumulative Index to Nursing and Allied Health; 1981 to May 2012) (Appendix 5), using the EBSCOHost platform;

  • Campbell Collaboration's Social, Psychological, Educational and Criminological Trials Register (C2-SPECTR http://geb9101.gse.upenn.edu/: searched May 2009 (website no longer accessible)) (Appendix 6);

  • Education Resource Information Centre (ERIC) 1966 to May 2009) (Appendix 7), using Dialog Datastar.

Searching other resources

We handsearched the reference lists of relevant publications and reviews to identify further trial reports (Horsley 2011) (Appendix 8). We also searched the abstracts of Society for Clinical Trials (SCT) meetings from 1980 to 2012, the Current Controlled Trials metaRegister of Controlled Trials (mRCT) (www.controlled-trials.com/mrct), the Cochrane Methodology Register (in The Cochrane Library to April 2012) and the World Health Organization (WHO) trials platform (apps.who.int/trialsearch). We conducted a survey of Clinical Trial Units in the UK to identify further eligible trials not identified through other sources and the review was presented at the Society for Clinical Trials 31st Conference in Baltimore, USA in May 2010 and advertised on the Conference notice board with the aim of identifying potentially eligible trials from outside the UK.

Data collection and analysis

Selection of studies

One review author (VB) selected potentially eligible trials from the titles and abstracts retrieved by the searches, using a predesigned study eligibility screening form. We were over inclusive when screening, 0.7% (168/24,304) of records identified were sent for screening to a second review author (GR), which is 23% (168/735) of all potentially eligible records identified. We obtained full-text papers and two review authors (VB, GR) reviewed potentially eligible trials for inclusion. We contacted study authors for electronic copies of papers that we could not access through library sources. We were able to obtain copies of all the potentially eligible papers that we wanted to screen. We resolved disagreements by discussion with a third review author (SS). When necessary, we sought information from the original investigators for potentially eligible trials where we wished to clarify eligibility.

Data extraction and management

One review author (VB) extracted data from eligible retention trial and associated host trial papers and a second review author (JT) checked the entries. We reached consensus on any disparities by discussion with a third review author (SS). Data extracted for the host trial were aim, setting, disease area, comparators, primary outcome, sample size calculation, inclusion exclusion criteria, sequence generation and allocation concealment, and numbers randomised to each group. For the embedded retention trial, we extracted data for onset in relation to the host trial, source of the sample, aim, primary outcome and type of follow-up. The retention strategy details included type, frequency and timing of administration method of randomisation, numbers randomised, included and retained at primary analysis, and data required for the risk of bias assessment.

Assessment of risk of bias in included studies

To assess the validity of each retention trial we judged them against the four domains of the Cochrane 'Risk of bias' tool (Higgins 2008a). To assess selection bias, we recorded how the allocation sequence was generated at study level and the methods used to conceal the allocation. We assessed performance bias by recording methods used to blind participants if considered appropriate to do so. For some interventions, participants could not be blinded to the intervention (e.g. where vouchers, cash or gifts were administered). However, in these cases, study personnel could be blinded to the allocation if administration of the intervention was carried out by someone unaware of the allocation.

As retention is the subject of our review, and retention of participants is the primary outcome, attrition from the trials does not constitute a bias and has not been included in the 'Risk of bias' tables. We assessed each included retention trial for selective outcome reporting by recording the primary outcome for the trial and the outcomes for which results were reported. A judgement was made about each trial for each risk of bias domain assessed. For completed host trials (within which retention trials were embedded), we only assessed sequence generation and allocation concealment, in order to ensure the host trial was randomised.

Measures of the effect of the methods

We calculated risk ratios (RR) and their 95% confidence intervals (CI) for retention to determine the effect of strategies on this outcome.

Unit of analysis issues

For retention trials that randomised individuals and clusters, the unit of analysis was the participant. For cluster randomised trials that ignored clustering in the analysis, we inflated the standard errors (SE) to avoid overprecise estimates of effect as follows (Higgins 2008b).

  1. We calculated the RR, 95% CI and SE based on participants in the usual way (i.e. ignoring clustering).

  2. This standard error was then inflated using the design effect to get an adjusted SE: adjusted SE = SE X√ design effect. With the design effect calculated as follows: design effect = 1 + (M - 1) ICC where M = mean cluster size, ICC = the intracluster correlation coefficient.

  3. Where published ICCs were not available, we used the mean ICC from appropriate external estimates for Land 2007. This was the mean of estimates for the return of EuroQol questionnaires (ICC = 0.054) from a source recommended by the Cochrane Handbook for Systematic Reviews of Interventions (Section 16.3.4) (Higgins 2008b) and www.abdn.ac.uk/hsru/documents/iccs-web.xls (last accessed 27 September 2013).

  4. We entered the effect estimate and the new updated SE into Review Manager 5 using the generic inverse variance (RevMan 2012).

Where the number of participants randomised was not clearly stated in the included study report, we contacted the study authors for this information.

Dealing with missing data

We contacted study authors for data for the risk of bias assessment, numbers randomised to each group and numbers retained in each group at the primary endpoint. We described outcomes with insufficient data qualitatively. For time-to-event outcomes, we used the time point of the host study primary outcome, taking account of censoring if necessary and if the data were available.

Assessment of heterogeneity

We measured heterogeneity of the intervention effect using the Chi2 statistic at a significance level of 0.10 and the I2 statistic (Higgins 2003), and explored through subgroup analyses.

Assessment of reporting biases

We would have investigated reporting bias using tests for funnel plot asymmetry if sufficient data had been available (Egger 1997; Sterne 2008).

Data synthesis

If there was no substantial heterogeneity, we pooled RRs using the fixed-effect model. If heterogeneity was detected and could not be explained by subgroup or sensitivity analyses, we used the random-effects model or did not pool results.

For factorial trials (Sharp 2006a-h, Kenton 2007a-d), all main effects were included as separate trial comparisons if they addressed different categories of strategies. Where the main effects addressed two or more strategies within the same category (e.g. Bowen 2000abc), we combined the relevant intervention groups and compared them with the control group. We also compared each intervention group with the control group, as separate trial comparisons, in exploratory analyses. For one 2 x 2 x 2 x 2 factorial trial (Renfroe 2002a-d), the numbers randomised for each group were not available at the time of analysis so comparison groups were collapsed as far as possible and then treated as separate trial comparisons in the appropriate analyses. For two three-armed trials that compared two similar intervention groups with one control group, we combined the intervention groups and compared it with the control group for the main analyses (Bauer 2004ab, Khadjesari 2011 1abc). We also compared each intervention group as separate trial comparisons in exploratory analyses.

These approaches allowed full exploration of the data and also avoided double counting and over-precise pooled estimates of effect in our main analyses. However, this also meant that there were occasionally a greater number of trial comparisons than trials.

Computations for the absolute benefits of effective strategies on questionnaire response and trial retention were based on absolute risk reductions derived from meta-analysis RRs (Cochrane Handbook for Systematic Reviews of Interventions, Section 12.5.4.2: Schünemann 2008).

Subgroup analysis and investigation of heterogeneity

To explore the effect of different strategies on trial retention, we planned the following subgroup analyses by the type of strategy used in included retention trials.

  • Whether the strategy was compared with usual follow-up or other strategies.

  • Whether in healthcare or non-healthcare settings.

  • Whether assessment of retention was immediate or longer term (e.g. if a response to a questionnaire was expected immediately or at later time points).

  • Whether the strategy was participant or management focused.

However, we identified such a diversity of retention trials and interventions that these analyses were inappropriate or not possible. Therefore, different types of strategies were analysed separately and new subgroups were defined within these before we conducted the analyses.

(a) Incentives

We subgrouped retention trials or trial comparisons evaluating the addition of an incentive strategy versus none as follows for analysis.

  1. Monetary incentives given upfront, defined as money given to the trial participant prior to data collection in cheque, cash or voucher format.

  2. Non-monetary incentives, defined as gifts, for example, pens or certificates.

  3. Offers of monetary incentives after data collection, defined as a promise of the incentive after return of outcome data through attendance for scheduled follow-up or receipt of follow-up questionnaires.

  4. Offers of non-monetary incentives defined as a promise of the non-monetary incentive after return of outcome data through attendance for scheduled follow-up or receipt of follow-up questionnaires.

We subgrouped retention trials or trial comparisons comparing different values of monetary incentives into:

  1. those offering incentives;

  2. those both giving and offering an incentive for any subsequent data (e.g. sending GBP5 with a questionnaire with an offer of GBP5 if the questionnaire is returned).

We analysed retention trials evaluating the addition of a monetary incentive versus either an offer of a monetary incentive or follow-up by telephone separately.

(b) Communication

We grouped retention trials or trial comparisons of the effect of different communication strategies into letter, post and reminder strategies for analysis as follows.

  1. Enhanced versus standard cover letter.

  2. Total design method versus standard postal communication strategy.

  3. Priority versus regular post.

  4. Additional reminders versus usual reminders to trial sites.

  5. Additional reminders versus usual follow-up to trial participants.

  6. Early versus late administration of questionnaire (i.e. sending questionnaires two to three weeks after a follow-up visit versus one to four months after a follow-up visit).

  7. Recorded delivery versus telephone reminder.

(c) Questionnaire structure

We subgrouped trials of questionnaire strategies into length of questionnaire, clarity of meaning, order of questions and layout as follows.

  1. Short versus long questionnaire.

  2. Long and clear questionnaire versus short and condensed questionnaire.

  3. Medical condition questions first versus generic questions first.

  4. Relevance of questionnaires: alcohol versus mental health questionnaires.

There were no subgroups for behavioural, case management and methodology retention trials.

Our analyses focused on the primary endpoint of retention. We initially pooled retention trials within subgroups using the fixed-effect model and quantified heterogeneity. We assessed whether these subgroups had a differential impact on retention using the test for interaction. We did not pool trials if results were inconsistent or heterogeneity was excessive.

Sensitivity analysis

To assess the robustness of the results we planned sensitivity analyses that excluded quasi-randomised retention trials.

Results

Description of studies

The studies are described in the Characteristics of included studies, Characteristics of studies awaiting classification, and Characteristics of excluded studies tables.

Results of the search

We identified 24,304 abstracts, titles and other records from database searches to May 2012, handsearches of reviews, lists of references in included papers, SCT conference abstracts (to 2012), personal contact with trialists, and the survey of UK Clinical Trials Units (Figure 1). We screened 735 full-text papers, reports and manuscripts for eligible studies. Of 68 potentially eligible studies, we found 30 to be subsequently ineligible. This left 38 retention trials for inclusion in the review. The retention trials were embedded in real trials (host trials). We identified 11retention trials from CENTRAL, MEDLINE and CINAHL; 14 from handsearching reviews, conference abstracts, and references lists of eligible papers; and 13 through personal communications or correspondence with clinical trials units. We evaluated six broad types of strategy to improve retention in randomised trials. Most strategies were targeted at increasing questionnaire response. The strategies used for this were incentives, communication, methodology and questionnaire design strategies. There was minimal evidence for the use of behavioural and case management strategies to improve retention.

Figure 1.

Attrition study flow diagram.

Included studies

Of the 38 eligible retention trials, 28 were published in full, one as an abstract (Kenton 2007a-d), and one as part of a PhD thesis (Nakash 2007). Four retention trial publications contained two trials each (Khadjesari 2011; McCambridge 2011; McColl 2003; Severi 2011). Eight retention trials are unpublished as of June 2013 (Bailey 1; Bailey 2; Edwards 2001; Land 2007; Letley 2000; MacLennan; Marson 2007; Svoboda 2001).

Host trials

Twenty-two host trials included a single retention trial (AVID investigators 1997; Boyd 2002; Chaffin 2009; Cooke 2009; Cox 2008; Gail 1992; Dennis 2009; Hughes 1984; International Stroke Trial Group 1997; Kenyon 2001; Lamb 2007; Leigh Brown 2001; Marson 2007 (2); Omenn 2006; Porterhouse 2005; Rothert 2006; Tai 1999; Tilbrook 2011; TOMBOLA 2009a; TOMBOLA 2009b; UK BEAM 2004). Two host trials from this group were unpublished (for the retention trials by Ashby 2011 and Land 2007).

The other host trials included multiple retention trials (one unpublished for the retention trials by Bailey 1 and Bailey 2). Two retention trials (Ford 2006; Subar 2001) were embedded in the US-based Prostate, Lung, Colorectal, Ovarian (PLCO) screening trial of Prorok 2000; two (Avenell 2004; MacLennan) in the RECORD fracture prevention trial (RECORD 2007); two (Edwards 2001; Svoboda 2001) in the CRASH trial (CRASH Trial collaborators 2004); four (Khadjesari 2011 1abc; Khadjesari 2011 2; McCambridge 2011 1; McCambridge 2011 2) in the Down your Drink Trial (Murray 2007); two (Bailey 1; Bailey 2) in a feasibility study for the Sex unzipped website (unpublished); two (Severi 2011 1; Severi 2011 2) in the Text to Stop smoking cessation trial (Free 2011); and two (McColl 2003 1; McColl 2003 2) in the COGENT trial (Eccles 2002).

Participants and settings

Included retention trials were conducted in a broad spectrum of clinical conditions and geographical settings (see Appendix 9). Eight included retention trials were embedded in trials for the treatment of alcohol and smoking dependency (Bauer 2004ab; Hughes 1989; Khadjesari 2011 1abc; Khadjesari 2011 2; McCambridge 2011 1; McCambridge 2011 2; Severi 2011 1; Severi 2011 2), and four in trials investigating treatments for injuries (Edwards 2001; Gates 2009; Nakash 2007; Svoboda 2001). Six retention trials were set in treatment trials for cancer, cardiovascular disease, epilepsy and back pain (Dorman 1997; Land 2007; Letley 2000; Man 2011; Marson 2007; Renfroe 2002a-d), and four were embedded in screening trials for cancer, postnatal depression, and elderly diseases (Ford 2006; Kenton 2007a-d; Sharp 2006a-h; Subar 2001). Seven retention trials were embedded in prevention trials, which included two cancer prevention trials for lung and breast cancer (Bowen 2000abc; Sutherland 1996), one migraine prevention trial (Ashby 2011), and three fracture prevention trials (Avenell 2004; Cockayne 2005; MacLennan). Four retention trials were conducted in clinical management trials for orthopaedics, asthma, diabetes and angina (Leigh Brown 1997; McColl 2003 1; McColl 2003 2; Tai 1997). Six retention trials were conducted in other areas: exercise (Cox 2008), parenting (Chaffin 2009), weight management (Couper 2007), neonatal medicine (Kenyon 2005), and sexual health promotion (Bailey 1; Bailey 2).

Twenty-five retention trials were UK based, nine were USA based and two were set in Canada. The remainder were set in Czech Republic and Australia (see Characteristics of included studies table).

Retention trials were embedded in host trials that recruited participants from different settings. Five trials recruited participants directly from the community. Sixteen trials were conducted through secondary care facilities. One trial recruited participants through a combination of state workers compensation programmes, occupational and physician clinic, a surveillance programme and union records. Six UK trials recruited solely through general practitioner (GP) practices and two used a combination of recruitment through GP practices and the media. Seven trials recruited participants via the Internet, six of these were UK based and the other was US based. For one US-based smoking cessation trial, it was unclear how participants were recruited (see Characteristics of included studies table).

Design of included retention trials

One trial was hosted in a clustered randomised trial and used this design to evaluate a strategy to improve retention (Land 2007). Four retention trials used different factorial designs (Bowen 2000abc; Kenton 2007a-d; Renfroe 2002a-d; Sharp 2006a-h). There was also one three-armed trial (Bauer 2004ab), and three four-armed trials (Khadjesari 2011 1abc; McCambridge 2011 1; McCambridge 2011 2).

Five trials used quasi-randomisation to allocate participants (Bowen 2000abc; Ford 2006; Gates 2009; McColl 2003 1; McColl 2003 2). Two used participant identification numbers (Ford 2006; Gates 2009), and two allocated the first half of a simple random sample of participants to receive one version of a questionnaire, while the remaining half was allocated to a second version (McColl 2003 1; McColl 2003 2). One retention trial used day of clinic visit to allocate participants (Bowen 2000abc).

All trials targeted individual trial participants, except one that targeted trial sites (Land 2007).

We recorded the timing of randomisation in the host trial versus the timing of randomisation in the retention trial. Four trials commenced during a randomised pilot study for the host trial (Khadjesari 2011 1abc; Letley 2000; McCambridge 2011 1; Sutherland 1996). One study started before the host trial (Chaffin 2009). Twenty-nine trials commenced during follow-up for the host trial (Ashby 2011; Avenell 2004; Bailey 1; Bailey 2; Bowen 2000abc; Cockayne 2005; Couper 2007; Cox 2008; Dorman 1997; Edwards 2001; Ford 2006; Gates 2009; Khadjesari 2011 2; Land 2007; Leigh Brown 1997; MacLennan; Man 2011; Marson 2007; McCambridge 2011 2; McColl 2003 1; McColl 2003 2; Nakash 2007; Renfroe 2002a-d; Severi 2011 1; Severi 2011 2; Sharp 2006a-h; Subar 2001; Svoboda 2001; Tai 1997). For one trial, it was unclear when the retention trial started in relation to the host trial (Kenton 2007a-d). Three retention trials started after the host trial had finished (Bauer 2004ab; Hughes 1989; Kenyon 2005): Kenyon 2005 followed-up seven-year-old children of mothers enrolled in the ORACLE trial (Kenyon 2001), Bauer 2004ab followed up participants in the COMMIT smoking cessation trial (Gail 1992), eight years after the original trial was completed and Hughes 1989 followed up participants in a smoking cessation trial six months after that study finished (Hughes 1984).

Strategies to improve retention

Retention in trials and response to questionnaires were the outcomes measured for all included trials. The included trials evaluated six different types of strategies to improve response or retention. Incentives, communication strategies, variation in questionnaire design, methodology strategies, and combinations of communication and incentive strategies evaluated improving response to postal and electronic questionnaires. Behavioural strategies, case management and some non-monetary incentives were used to encourage participants to return to trial sites for follow-up visits. Each type of strategy is described separately below.

Outcome measures in the included trials

Thirty-four retention trials measured response to questionnaires. Among these, the questionnaires were by post in 26 trials, electronically in four and one was done by interview. For another three retention trials, response was return of biomedical kits or biomedical kits plus a questionnaire (see Characteristics of included studies table).

Four included trials measured the number of participants remaining in the trial (Bowen 2000abc; Chaffin 2009; Cox 2008; Ford 2006)

Ten included trials specified that their primary outcome was questionnaire response at a particular time point: McCambridge 2011 1 measured response at one and three months, McCambridge 2011 2 measured response at three and 12 months, and Khadjesari 2011 1abc and Khadjesari 2011 2 measured response within 40 days of the first reminder. For Severi 2011 1, the primary outcome was completed follow-up at 30 weeks from randomisation, Severi 2011 2 used return of specimens one month after a telephone call, Avenell 2004 used retention at one year measured by questionnaire return but also reported retention at four and eight months. Cockayne 2005 and Sharp 2006a-h had final follow-up questionnaire response at any time as their primary outcome.

Two included trials reported questionnaire response at one time point only but without specifying that this was the primary outcome for the trial (Edwards 2001; Svoboda 2001). These trials measured response at three months from the questionnaire being sent. One trial reported trial retention at one time point only (three years) but without specifying that this was the primary outcome for the trial (Ford 2006). This was measured as completing the next cancer screening in a cancer screening trial. In each of these three trials, we used these data for analyses.

Two trials recorded questionnaire response at two time points without stating which was the primary outcome (Dorman 1997; Gates 2009). One trial recorded retention at two time points without stating which was the primary outcome (Cox 2008). We used data for response/retention after the first contact with respondents as the primary outcome for analyses. One trial reported response at three time points (4 weeks, 12 weeks and 9 months), which were all stated as the primary outcome (Nakash 2007). We used the data for week four in our main analysis.

Five trials reported data in survival curves. For these, we used the final analysis point (Ashby 2011; Bowen 2000abc; Chaffin 2009; Land 2007; Sutherland 1996). Authors confirmed data when it had been extracted. Fifteen trials reported the number of questionnaires returned with no time point specified (Bauer 2004ab; Couper 2007; Hughes 1989; Kenton 2007a-d; Kenyon 2005; Leigh Brown 1997; Letley 2000; MacLennan; Man 2011; Marson 2007; McColl 2003 1; McColl 2003 2; Renfroe 2002a-d; Subar 2001; Tai 1997).

Addition of incentive versus none

There were 14 retention trials of incentives and 19 trial comparisons (Table 1). Thirteen trials were aimed at improving questionnaire response in trials and one trial was aimed at improving return for follow-up at trial site (Bowen 2000abc). The different incentive strategies aimed at improving questionnaire response were vouchers, cash, a charity donation, entry to prize draws, cheques, a certificate of appreciation and offers of study results. Incentive strategies aimed at improving retention were: certificates of appreciation and lapel pins. The value of incentives used in UK evaluations ranged from GBP5 to GBP20 and were in cash, cheque or voucher format. The value of incentives used in US-based studies was USD2 to USD10. For offers of entries into prize draws, the values were higher, ranging from GBP25 to GBP250 for UK prize draws and USD50 for US-based prize draws. One trial evaluated giving a monetary incentive with a promise of a further incentive for return of trial data (Bailey 2).

Table 1. Trials evaluating incentive strategies
Trial or trial comparison Incentive groups Control group Outcome type
Addition of incentive vs. none
Bauer 2004ab

a) USD10 cheque

b) USD2 cheque

Arms combined for main analysis

No incentiveDNA specimen kit return plus postal questionnaire response 
Gates 2009GBP5 voucherNo incentivePostal questionnaire response
Kenyon 2005GBP5 voucherNo incentivePostal questionnaire response
Khadjesari 2011 1ac

a) Offer GBP5 voucher

c) Offer of entry into GBP250 prize draw, groups combined for main analysis

No incentiveInternet-based questionnaire response
Khadjesari 2011 2Offer of GBP10 Amazon.co.uk voucherNo incentiveInternet-based questionnaire response
Bowen 2000abc

a) Certificate

b) Pin

c) Pin and certificate groups combined for main analysis

No incentiveParticipants retention
Renfroe 2002a  Certificate of appreciationNo certificate of appreciationPostal questionnaire response
Sharp 2006aPenNo penPostal questionnaire response
Sharp 2006bPenNo penPostal questionnaire response
Sharp 2006cPenNo penPostal questionnaire response
Sharp 2006dPenNo penPostal questionnaire response
Cockayne 2005Offer of study resultsNo offerPostal questionnaire response
Hughes 1989Offer of free reprint of resultsNo offerPostal questionnaire response
Khadjesari 2011 1bOffer of GBP5 charity donationNo offerInternet-based questionnaire response
Addition of monetary incentive to both groups
Bailey 1 unpublishedOffer of GBP20 shopping voucherOffer of GBP10 shopping voucherPostal questionnaire response
Bailey 2 unpublishedShopping voucher: GBP10 in advance and GBP10 on data returnShopping voucher: GBP5 in advance and GBP5 on data returnQuestionnaire response and chlamydia kit return
Addition of monetary incentive vs. offer of incentive
Kenton 2007aUSD2 coinDraw for USD50 gift voucherPostal questionnaire response
Kenton 2007bUSD2 coinDraw for USD50 gift voucherPostal questionnaire response
Offer of prize draw vs. no offer
Leigh Brown 1997Aware of monthly prize draw of GBP25 gift voucherNo offer of drawPostal questionnaire response
Communication strategies

There were 14 retention trials of communication strategies to improve response to postal questionnaires or return of biomedical test kits, or both, in randomised trials. There were 20 trial comparisons (Table 2). Strategies evaluated were: enhanced letters, additional reminders to participants, priority mailing of questionnaires, time of questionnaire administration, telephone contact and reminders to trial sites of upcoming assessments. One trial used a combination of postal communication strategies known as the total design method (TDM) (Sutherland 1996). This included sending letters in a white envelope with a hospital logo and commemorative stamp, a hand-signed letter on headed notepaper, with a reply self addressed stamped envelope, enclosing the contents. Follow-up was with a postcard sent after seven days followed by two reminder letters. This was compared with a customary method for postal follow-up. One trial evaluated the addition of an electronic SMS (short message service) text reminder on the day participants were due to receive their postal questionnaire (Man 2011).

Table 2. Trials evaluating communication strategies
  1. AVID: Antiarrhythmics Versus Implantable Defibrillators; SMS: short message service.

Trial or trial comparison Communication strategy Control arm Outcome type
Enhanced letter vs. standard letter
Renfroe 2002cCover letter signed by physicianCover letter signed by co-ordinatorPostal questionnaire response
Marson 2007Letter explaining the approximate length of time to complete questionnaireStandard letterPostal questionnaire response
Total design method vs. customary method
Sutherland 1996Total design method for postal follow-upStandard method for postal follow-upPostal questionnaire response
Priority vs. regular post
Renfroe 2002bExpress deliveryStandard deliveryPostal questionnaire response
Sharp 2006eDespatch first-class stampDespatch second-class stampPostal questionnaire response
Sharp 2006fDespatch first-class stampDespatch second-class stampPostal questionnaire response
Sharp 2006gSecond-class return envelopeFree post return envelopePostal questionnaire response
Sharp 2006hSecond-class return envelopeFree post return envelopePostal questionnaire response
Kenton 2007cPriority mailStandard mailPostal questionnaire response
Kenton 2007dPriority mailStandard mailPostal questionnaire response
Additional reminder vs. usual follow-up
Ashby 2011Electronic reminder No electronic reminder                          Postal questionnaire response
MacLennan unpublishedTelephone reminderNo telephone reminderPostal questionnaire response
Nakash 2007Trial calendar given at recruitment with questionnaire due datesNo calendarPostal questionnaire response
Severi 2011 1Text message and fridge magnet both emphasising social benefits of study participation.Text message reminder sent 3 days after questionnairePostal questionnaire response
Severi 2011 2Telephone reminder from principle investigatorStandard procedures.Return of cotinine samples
Man 2011SMS text message as follow-up questionnaire sent outNo SMS text messagePostal questionnaire response
Additional trial site reminder vs. usual reminder
Land 2007Prospective monthly reminder of upcoming assessments to trial sitesNo extra reminder to trial sitesPostal questionnaire response
Early vs. late administration of questionnaire
Renfroe 2002dQuestionnaire sent 2-3 weeks after last AVID follow-up visitQuestionnaire sent 1-4 months after last AVID follow-up visitPostal questionnaire response
Recorded delivery vs. telephone reminder
Tai 1997Recorded delivery reminderTelephone reminderPostal questionnaire response
Addition telephone follow-up vs. incentive
Couper 2007Telephone survey by trained interviewerPostal questionnaire and USD5 billPost and questionnaire response

Five trials evaluated a combination of communication strategies and incentives to improve retention from randomised trials (Couper 2007; Kenton 2007a-d; Renfroe 2002a-d; Sharp 2006a-h). The communication strategies were; first- and second-class outward post (Kenton 2007a-d; Renfroe 2002b; Sharp 2006a-h), stamped and business reply envelopes (Sharp 2006a-h), letters signed by different study personnel (Renfroe 2002c), letters posted at different times (Renfroe 2002d), text messages (Man 2011; Severi 2011 1), and a telephone survey (Couper 2007).

Questionnaire format

The effect of a change in questionnaire format on response to randomised trial questionnaires was evaluated in eight trials with 10 comparisons (Table 3). Formats evaluated were questionnaire length: short versus long (Dorman 1997; Edwards 2001; McCambridge 2011 1b; McCambridge 2011 2b; Svoboda 2001), long and clear versus short and condensed (Subar 2001), and the order of questions (Letley 2000; McColl 2003 1; McColl 2003 2).

Table 3. Trials evaluating new questionnaire strategies
  1. APQ: Alcohol Problems Questionnaire; AUDIT: Alcohol Use Disorders Identification Test; LDQ Leeds Dependency Questionnaire; PLCO: Prostate, Lung, Colorectal, Ovarian; SF-36: Short Form 36 item.

Trial or trial comparison Questionnaire strategy Control arm Outcome type
Short vs. long
Dorman 1997Short EuroQolLong SF-36 questionnaire.Postal questionnaire response
Edwards 2001 unpublished1-page, 7-question functional dependence questionnaire 3-page, 16-question functional dependence questionnairePostal questionnaire response
Svoboda 2001 unpublished1-page, 7-question functional dependence questionnaire 3-page, 16-question functional dependence questionnairePostal questionnaire response
McCambridge 2011 1b

AUDIT Short

+

LDQ

APQInternet-based questionnaire response
McCambridge 2011 2b

AUDIT Short

+

LDQ

APQInternet-based questionnaire response
Long and clear vs. short and condensed
Subar 2001DHQ (36-page food frequency questionnaire)PLCO (16-page food frequency questionnaire)

Postal questionnaire response and onsite completion  

 

Question order
McColl 2003 1

Asthma condition-specific questions first followed by generic

 

Generic questions followed by condition specific

Postal questionnaire

response

McColl 2003 2Angina condition-specific questions followed by genericGeneric questions followed by condition specificPostal questionnaire response
Letley 2000 unpublishedRDQ at front and SF-36 at backSF-36 at front RDQ at backPostal questionnaire response
Relevance of questionnaire
McCambridge 2011 1aAPQ23 itemsCORE-OM Mental health assessment 23/34 itemsInternet-based questionnaire response
McCambridge 2011 2a

AUDIT Short

+

LDQ

CORE-OM Mental health assessment 10 itemsInternet-based questionnaire response

Two further included trials evaluated the effect of the relevance of a questionnaire on response (McCambridge 2011 1a; McCambridge 2011 2a). Relevance was defined as assessing alcohol problems rather than mental health in the context of an Internet-based intervention for hazardous drinkers (McCambridge 2011 1; McCambridge 2011 2).

Behavioural strategies

There were two trials of behavioural strategies used for retention in randomised trials (Chaffin 2009; Cox 2008). Cox 2008 compared motivational workshops versus information sheets. Chaffin 2009 compared self motivation orientation versus standard information in the context of a parenting programme. In this case, the retention trial was run prior to the host trial with the intention of improving retention in the subsequent parenting programme evaluation trial. The analysis was based on the number eligible for inclusion in the primary analyses for the subsequent parenting programme because we do not know the allocation of those who dropped out between first and second randomisations. Complete time-to-event data were not available for Chaffin 2009, but, as only two participants were censored in the analysis, this is unlikely to have biased the results.

Case management

One retention trial evaluated the effect of intensive case management procedures on retention of African American male participants in a cancer screening trial (Ford 2006).

Methodology strategies

One included trial used a trial design where people knew which treatment they received. The trial compared questionnaire response in an open versus blind trial (Avenell 2004).

Studies excluded from analyses

Two eligible trials could not be included in the analysis (Leigh Brown 1997; Letley 2000). Host trial participants in the retention trial by Leigh Brown 1997 were divided into two groups; one randomised, the other determined by preference of the referring primary care practitioner. The author confirmed that participants in the retention trial were from both randomised and non-randomised groups of the host trial and that these could not be separated.

One recently completed, unpublished trial that is not included in the review examined the effect of newsletters on retention (Mitchell). This trial will be included in the review update.

Excluded studies

See Characteristics of excluded studies table.

We excluded trials because they were either part of a non-randomised host study, or they were not a randomised retention trial, or the primary outcome was type of data item missingness. Other excluded trials were aimed at increasing treatment compliance or baseline questionnaire response. We contacted investigators to confirm aspects of eligibility.

Risk of bias in included studies

See Characteristics of included studies.

Allocation

All included retention trials reported that participants were randomly allocated to groups for comparison. Twenty-four included trials described adequate sequence generation by a computerised random number generator, block randomisation or use of a table of random numbers table (Avenell 2004; Bailey 1; Bailey 2; Bowen 2000abc; Chaffin 2009; Cockayne 2005; Cox 2008; Hughes 1989; Kenyon 2005; Khadjesari 2011 1abc; Khadjesari 2011 2; Land 2007; Leigh Brown 1997; Letley 2000; MacLennan; Man 2011; Marson 2007; McCambridge 2011 1; McCambridge 2011 2; Nakash 2007; Renfroe 2002a-d; Severi 2011 1; Severi 2011 2; Sutherland 1996). There was insufficient information about the sequence generation for 10 included trials, these were all described as randomised in the retention trial publications (Ashby 2011; Bauer 2004ab; Couper 2007; Dorman 1997; Edwards 2001; Kenton 2007a-d; Sharp 2006a-h; Subar 2001; Svoboda 2001; Tai 1997). Five included trials used quasi-randomisation to allocate participants (Bowen 2000abc; Ford 2006; Gates 2009; McColl 2003 1; McColl 2003 2).

Several methods were used to avoid foreseen allocation of participants; sequence generation by a trial statistician and implemented by a trial manager; sequence generation by an independent researcher, a central randomisation service, or by a nurse using a preprogrammed computer; or allocation by sealed envelopes or sequentially numbered packs. Fifteen trials reported both adequate sequence generation and allocation concealment (Avenell 2004; Bailey 1; Bailey 2; Cockayne 2005; Cox 2008; Hughes 1989; Kenyon 2005; Khadjesari 2011 1abc; Khadjesari 2011 2; Letley 2000; MacLennan; Man 2011; McCambridge 2011 1; McCambridge 2011 2; Nakash 2007).

Blinding

Blinding of participants was generally not possible in included trials. For example, it is not possible to blind participants to the following strategies to increase trial retention or response to questionnaires: incentive or offer of incentive, behavioural (Cox 2008), or case management strategies (Ford 2006), different types of communication strategies, or questionnaire format strategies. In a number of trials, authors mentioned that participants were aware of the intervention they were getting but were unaware that this was being evaluated (Bowen 2000abc; Chaffin 2009; Kenton 2007a-d; Kenyon 2005; Leigh Brown 1997; MacLennan; Marson 2007; McColl 2003 1; McColl 2003 2). For other trials, blinding of participants or study personnel to the outcome or intervention was not reported. For one trial, a judgement about blinding was not applicable because the study evaluated the effect of blind versus open trials on retention (Avenell 2004).

Incomplete outcome data

The primary outcome measure for this review was retention, and this was well reported. We contacted authors for clarification of any exclusions after randomisation if this was unclear from retention trial reports.

Selective reporting

Although retention trial protocols were not available for included trials, the included published and unpublished papers reported all expected outcomes for retention.

Other potential sources of bias

There were few other potential sources of bias identified from reports of included retention trials. For the behavioural trial by Cox 2008, the authors identified that the "walk and swim sessions were not separated according to the behavioural intervention. Participants were asked not to discuss written materials in the practical sessions". Therefore, potential contamination between study groups could have led to biased results.

Effect of methods

1. Incentive strategies

There were 14 trials of incentives giving 19 trial comparisons with 16,253 participants. There was considerable heterogeneity across incentive subgroups (P value < 0.00001) (Analysis 1.1), so we decided not to pool the results for incentives.

Addition of incentive

The three trials (3166 participants) that evaluated the effect of giving monetary incentives to participants showed that the addition of monetary incentives was more effective than no incentive at increasing response to postal questionnaires (RR 1.18; 95% CI 1.09 to 1.28, P value < 0.0001) (Analysis 1.1). A sensitivity analysis excluding the quasi-randomised trial by Gates 2009 still showed that the addition of a monetary incentive remained more effective than none (RR 1.31; 95% CI 1.11 to 1.55, P value = 0.002) (Analysis 2.1).

Based on two Internet-based trials (3613 participants), an offer of a monetary incentive promoted greater return of electronic questionnaires than no offer (RR 1.25; 95% CI 1.14 to 1.38, P value < 0.00001; heterogeneity P value = 0.14) (Analysis 1.1). However, a single trial comparison suggested that an offer of a monetary donation to charity did not increase response to electronic questionnaires (RR 1.02; 95% CI 0.78 to 1.32; P value = 0.90) (Analysis 1.1).

Based on six trials (6322 participants), there was no clear evidence that the addition of non-monetary incentives improved questionnaire response (RR 1.00; 95% CI 0.98 to 1.02, P value = 0.91), but there was some heterogeneity (P value = 0.02) (Analysis 1.1). A sensitivity analysis excluding the quasi-randomised trial (Bowen 2000abc) showed a similar effect (RR 1.00; 95% CI 0.93 to 1.08, P value = 0.99) (Analysis 2.1) and heterogeneity (P value = 0.01).

Two trials (1138 participants) evaluating offers of non-monetary incentives suggest that an offer of a non-monetary incentive is neither more nor less effective than no offer (RR 0.99; 95% CI 0.95 to 1.03, P value = 0.60) at improving questionnaire response (Analysis 1.1).

In exploratory analyses, the different incentive arms that were combined for the main analysis did not appear to show differential effects (Analysis 3.1).

Addition of monetary incentive to both study arms

Two trials (902 participants) show that higher value incentives are better at increasing response to postal questionnaires than lower value incentives (RR 1.12; 95% CI 1.04 to 1.22, P value = 0.005) irrespective of how they are given (Analysis 5.1).

Addition of monetary incentive versus offer of a monetary incentive

Two trials (297 participants) provided no evidence that giving a monetary incentive is better than an offer of entry into a prize draw for improving response to postal questionnaires (RR 1.04; 95% CI 0.91 to 1.19, P value = 0.56) Analysis 6.1.

Addition of an offer of entry into a prize draw versus none

We excluded one trial from the analysis (Leigh Brown 1997). The results showed higher responses in the group offered entry into a prize draw compared with the group not offered entry into the draw (70.5% versus 65.8%).

2. Communication strategies

There were 14 trials of communication strategies and 20 comparisons with 9822 participants.

Addition of telephone survey versus monetary incentive plus questionnaire

One trial (700 participants) showed no clear evidence that a telephone survey was either more or less effective than a monetary incentive and a questionnaire for improving response (RR 1.08; 95% CI 0.94 to 1.24, P value = 0.27) (Analysis 4.1).

Enhanced versus standard letters

Results from two trials (2479 participants) showed that an enhanced letter was neither more nor less effective than a standard letter for increasing response to trial postal questionnaires (RR 1.01; 95% CI 0.97 to 1.05, P value = 0.70) (Analysis 7.1).

Total design method versus customary method

Although based on a single trial (226 participants) the TDM package was more effective than a customary postal communication method at increasing questionnaire return (RR 1.43; 95% CI 1.22 to 1.67, P value < 0.0001) (Analysis 8.1).

Priority versus regular post

Based on the relevant arms of seven trials (1888 participants), there was no clear evidence that priority post was either more or less effective than regular post at increasing trial questionnaire return (RR 1.02; 95% CI 0.95 to 1.09, P value = 0.55) (Analysis 9.1).

Additional reminder versus usual follow-up practices

Six trials (3401 participants) evaluated the effect of different additional types of reminders to participants on questionnaire response. There was no evidence that a reminder was either more or less effective than no reminder at improving trial questionnaire response (RR 1.03; 95% CI 0.99 to 1.06, P value = 0.13) (Analysis 10.1).

Additional reminder to trial site versus usual reminder

Based on one cluster randomised trial (272 participants), a monthly reminder to trial sites of upcoming assessment was neither more nor less effective than the usual follow-up (RR 0.96; 95% CI 0.83 to 1.11, P value = 0.57) (Analysis 11.1).

Early versus late questionnaire administration

Based on one trial (664 participants), there was no clear evidence that sending questionnaires early either increased or decreased response (RR 1.10; 95% CI 0.96 to 1.26, P value = 0.19 (Analysis 12.1).

Recorded delivery versus telephone reminder

One small trial (192 participants) found that recorded delivery was more effective than a telephone reminder (RR 2.08; 95% CI 1.11 to 3.87; P value = 0.02) (Analysis 13.1).

3. New questionnaire strategies

New versus standard questionnaire

Eight trials with 10 comparisons (21,505 participants) evaluated the effect of a new questionnaire format on questionnaire response. Although there was some heterogeneity between the questionnaire subgroups (P value = 0.11) (Analysis 14.1), it did not seem reasonable to pool the results based on such different interventions.

Five trials (7277 participants) compared the effect of short questionnaires versus long on postal questionnaire response. There was only a suggestion that short questionnaires may be better (RR 1.04; 95% CI 1.00 to 1.08, P value = 0.07) (Analysis 14.1).

Based on one trial (900 participants; Subar 2001), there is no evidence that long and clear questionnaires were either more or less effective than shorter condensed questionnaires for increasing trial questionnaire response (RR 1.01; 95% CI 0.95 to 1.07, P value = 0.86) (Analysis 14.1).

Two trials (9435 participants; McColl 2003 1; McColl 2003 2) found no evidence that placing disease/condition questions before generic questions is either a more or less effective strategy than a generic questions before disease/condition questions strategy at increasing trial questionnaire response (RR 1.00; 95% CI 0.97 to 1.02, P value = 0.75) (Analysis 14.1). It should be noted that these were quasi-randomised trials (Analysis 15.1).

One trial in this category was not included in the analysis by Letley 2000, outcome data were not available for each study arm when this review was submitted and the overall response rate for this trial was 87%.

In the context of research on reducing alcohol consumption, there was also evidence that more relevant questionnaires (i.e. those relating to alcohol use) increased response rates (RR 1.07; 95% CI 1.01 to 1.14, P value = 0.03).

4. Behavioural/motivational strategies

Two community-based trials (273 participants; Chaffin 2009; Cox 2008) showed no evidence that the behavioural/motivational strategies used are either more or less effective than standard information for retaining trial participants (RR 1.08; 95% CI 0.93 to 1.24, P value = 0.31) (Analysis 16.1).

5. Case management

One trial (703 participants; Ford 2006) evaluated the effect of intensive case management procedures on retention. There is no evidence that intensive case management was either more or less effective than usual follow-up in the population examined (RR 1.00; 95% CI 0.97 to 1.04, P value = 0.99) (Analysis 17.1).

6. Methodology strategies

One fracture prevention trial (538 participants; Avenell 2004) evaluated the effect of participants knowing their treatment allocation (open trial) compared with participants blind/unaware of their allocation on questionnaire response. Using a trial design where people know which treatment they will receive led to higher questionnaire response rates (RR 1.37; 95% CI 1.16 to 1.63, P value = 0.0003) (Analysis 18.1).

Reporting bias

Although we planned to investigate potential reporting bias, there were too few studies in most strategies to allow formal testing. However, we were able to obtain considerable data from unpublished trials and those published with limited information, reducing the risk of such biases.

Absolute benefits of strategies to improve retention

The absolute benefits of effective strategies on questionnaire response are illustrated in Table 4. The baseline response rates were broadly typical of the response rates seen in trials. The number of questionnaires returned were based on the assumed control arm risk.

Table 4. Gain in number of questionnaires returned per 1000 questionnaires sent
  1. RR: risk ratio.

Example of proportion of questionnaires

returned in control arm 

   30% 40% 50% 60% 70% 80% 90%
Strategy to improve retention  RR

1

RR

             
Addition of monetary incentive versus none1.180.847107927661532
Addition of offer of monetary incentive/prize draw versus none1.250.80014012010080604020
Addition of higher value monetary incentive versus addition of lower amount1.120.89077665544332211

Based on a 40% baseline response rate for postal questionnaires, the addition of a monetary incentive was estimated to increase response by 92 questionnaires per 1000 sent (95% CI 50 to 131).

With the addition of an offer of a monetary incentive in an Internet-based trial, based on a baseline response rate of 30%, trialists could expect an increase of 140 questionnaires per 1000 (95% CI 86 to 193).

For trials hoping to increase the return of postal questionnaires with chlamydia test kits, the number of kits returned was estimated to increase by 33 per 1000 sent when GBP20 was offered as an incentive, rather than GBP10 (95% CI 11 to 54).

Discussion

Summary of main results

Thirty-eight randomised retention trials were included in this review, evaluating six broad types of strategies to increase questionnaire response and retention in randomised trials. In 34 trials, strategies for increasing response to questionnaires were: incentives, communication strategies, new questionnaire format and methodological interventions. Four trials evaluated strategies to improve retention, these were: participant case management, behavioural and non-monetary incentive strategies. Trials were conducted across a spectrum of disease areas, countries, healthcare and community settings.

Strategies with the clearest impact on questionnaire response were: addition of monetary incentives compared with no incentive for return of postal questionnaires, addition of an offer of a monetary incentive when compared with none for return of electronic questionnaires, and an offer of GBP20 vouchers when compared with GBP10 for return of postal questionnaires and biomedical test kits. The evidence was less clear about whether shorter questionnaires rather than longer questionnaires increased response. The evidence was also less clear whether in the context of research on reducing alcohol consumption more relevant questionnaires increased response.

The addition of a non-monetary incentive, an offer of a non-monetary incentive compared with no incentive or, 'enhanced' letters, letters delivered by priority post, or additional reminders compared with standard communication strategies did not increase or decrease trial questionnaire response. Questionnaire structure also did not seem to increase response.

Although each was based on the results of a single trial, recorded delivery (proof of posting and an electronic copy of the signature available online) of questionnaires seemed more effective than telephone reminders, and a 'package' of postal communication strategies with reminder letters appeared better than standard procedures. A trial design where participants knew which treatment they were to receive also appeared more effective than a trial design where they were unaware of the treatment they were about to receive for return of questionnaires in a fracture prevention trial. Further evaluation of these strategies may be needed. Posting questionnaires early, questionnaire order, offers of charity donations or sending reminders to trial sites did not improve response.

Many trial outcome measures were collected using questionnaires, therefore, if response rates can be increased, retention will also be improved. No strategy had a clear impact on increasing the number of participants returning to trial sites for follow-up visits.

Overall completeness and applicability of evidence

The addition of a GBP5 voucher to usual follow-up procedures was effective for return of postal questionnaires in trials conducted between 2005 and 2009. The more recent unpublished studies by Bailey 1 Bailey 2 found GBP20 vouchers were more effective than GBP10 vouchers for return of postal questionnaires. Splitting the monetary incentive into money given before and after receipt of data could be more effective as a strategy to increase questionnaire follow-up with different population groups and in different trial settings where questionnaire response is low (e.g. with hard to reach groups that may include young male healthy adults, teenagers or residents in areas of high economic deprivation). This could be a cost-effective strategy because if questionnaires are not returned then money is saved. The value of the monetary incentive should not be so high as to be perceived as payment for data but more as an appreciation for efforts made by participants. Offering monetary incentives may increase the number of questionnaires returned per 1000 participants by at least as much as giving monetary incentives and giving higher valued monetary incentives, but has only been tested in online questionnaires. Offers of monetary incentives were also an effective strategy in the context of an online electronic questionnaire. These could be less costly to increase retention than the addition of a monetary incentive as only those who return the data are reimbursed. This would need further evaluation as the results were based on two Internet-based trials. It would be beneficial for trialists to know which is more effective: an offer of a monetary incentive or an upfront monetary incentive. We did not find any trials that made this direct comparison.

Shorter postal questionnaires have wide applicability to trials and could be considered as a useful strategy to increase trial questionnaire response in online Internet-based trials but there is only a suggestion that these are effective.

Several strategies showed no clear effect. The addition of non-monetary incentives in the form of pens, lapel pins and certificates of appreciation, or offers of non-monetary incentives through offering study results did not increase response or retention. A possible explanation might be how these items are valued by participants, or how they perceive their time is valued. Nevertheless, this result has the potential to reduce trial costs because associated saving could be channelled towards monetary incentives that have been shown to be effective.

The evidence showed that priority post (first-class post or equivalent) did not increase response. It is expensive as a means of communicating with participants and savings can be made by using regular (2nd-class) post instead.

Additional reminders sent to non-responders or as questionnaires were posted; enhanced letters, that is, letters signed by the principal investigator, or letters further explaining the anticipated length of time to complete a questionnaire, were not effective strategies to increase response. Enhanced letters and different types of additional reminders are used by trialists in current research practice. Too many reminders could be counterproductive to improving retention in randomised trials and details of the time expected to undertake specific tasks might be informative but off putting for participants. Nevertheless, letters and reminders are part of the research process and play a role in participant engagement especially if there is little face-to-face contact or in trials with long intervals between data collection time points.

Several strategies to increase questionnaire response need further evaluation to determine their effect but there is only a suggestion that these were effective. If participants are well and engaged with a trial, questionnaire length may not impact on response rates because participants may be happy to feedback on their condition in this way. For other conditions, for example, cancers and terminal illnesses, trial participants might prefer shorter questionnaires if their symptoms are problematic. Telephone follow-up compared with monetary incentive sent with a questionnaire needs further evaluation possibly with a cost-benefit analysis, as both could be expensive in time and human resources. Although appearing very effective, the total design method for postal questionnaires could be labour intensive to implement, expensive and may no longer be applicable to some participant groups (e.g. young people), or in trials using email, text or the Internet to collect data. Recorded delivery could be useful to ensure trial follow-up supplies reach their intended destination (e.g. biomedical specimen kits and questionnaires). Careful planning of day, date and time of delivery with each participant to avoid inconvenience might be necessary but again this strategy has the potential to be burdensome for trial co-ordinating centres and trial sites to administer. While trialists are assured that follow-up supplies are delivered with this strategy, participants might have the added burden of an extra visit to collect supplies from a sorting post office and this could be costly.

The use of open trials to increase questionnaire response can only be applied to trials where blinding is not required and could be counterproductive if a participant or clinician has a treatment preference. Bias associated with loss to follow-up resulting from these preferences could be avoided in blind trials.

Evaluations of strategies that encourage participants to return to trial sites for follow-up visits and monitoring were fewer than strategies to increase response to postal and electronic questionnaires, without further evidence case management and behavioural strategies cannot be recommended for use to encourage participant return.

This review identified no trials from low-income countries. All included studies were conducted in higher-income countries. Therefore, the strategies to increase retention identified by this review may not be generalisable to trials conducted in low-income countries because the interventions identified might not be socially, culturally or economically appropriate for trials run in these regions. The results may also not be applicable to all social groups as we were unable to examine response/retention by social characteristics such as economic disadvantage and social class. Most of the evidence in this review relates to increasing questionnaire follow-up in randomised trials for either the primary or secondary outcome for the host trial. The diversity between strategies and insufficient numbers in each of these categories meant that we could not do subgroup analyses by trial setting and disease area as planned.

Quality of the evidence

The extent of unpublished trials evaluating retention strategies is unknown; however, this review includes several unpublished trials and we made an effort to capture UK-based unpublished trials through our survey and research contacts. For some comparisons, results were based on one or two trials in a particular context. The inclusion of any further published and unpublished trials in future updates would improve the precision of the results of this review.

The six types of strategies that we identified targeted retention of trial participants in randomised trials. We believe response and retention were the relevant dichotomous outcomes to be reported for this review. Many other strategies used by trialists in practice to reduce attrition/increase response or retention in trials were not identified by this review (e.g. social support strategies; child care, Loue 2008; family support, De Sousa 2008; reduction in the number of visits, Schulz 2002). Evaluations of trial management strategies are also under-represented in the review (e.g. evaluations of site-specific reports, El Khorazaty 2007; levels of contact by the co-ordinating centre, Senturia 1998; training project staff).

Both published and unpublished included retention trials were fairly well conducted but could be improved. Five of the 39 trials included in the review were quasi-randomised. The motivation for conducting many of the included retention trials was reactive rather than planned upfront (i.e. when loss to follow-up became a problem during trial follow-up, rather than planned prior to host trial commencement).

Most trials used appropriate methods for randomisation or at least stated that they were randomised. For trials that did not describe their methods well or provide further information, there remains a potential risk of selection bias. Sensitivity analyses excluding quasi-randomised trials did not affect the results. In this context, where motivating participants to provide data or attend clinics is often the target of the interventions and so appropriately influences the outcome, lack of blinding is less of a concern. Retention is the outcome and was obtained for all but two trials, so similarly, attrition and selective outcome reporting bias are unimportant. Although the retention trials were fairly well conducted, they could be improved and they were often poorly reported. This may be because they were designed when loss to follow-up became a problem in a trial, rather than preplanned prior to host trial commencement.

Potential biases in the review process

Many words are used to describe loss to follow-up, for example, attrition, withdrawal and questionnaire non-response. We included these in our search strategy. We attempted to obtain unpublished trials and data by contacting authors and writing to UK clinical trials units and presenting at national and international conferences. We are confident that we have captured most studies and the spectrum of strategies that have been evaluated to date. It is conceivable, however, that less well-reported, ongoing, unpublished trials or trials conducted outside of the UK might have been missed. Most trials used appropriate methods for sequence generation or at least stated that they were randomised and concealed allocation. There is small risk that those that did not describe their methods well or provide further information did not use adequate methods for allocation and concealment and may have biased the results. However, sensitivity analyses excluding quasi-randomised trials did not affect the results. Blinding is hard to achieve in this context, where motivating participants to provide data or attend clinics is often the target of the interventions and so appropriately influences the outcome. 

Agreements and disagreements with other studies or reviews

The strategies that improve retention are, in some cases, the same as or similar to those found to be effective for cohort and cross-sectional study designs. However, prior to our review, it was not clear which of these strategies could be extrapolated to randomised trials. Successful retention strategies used in other study designs may be effective in trials settings and should be tested. Edwards' review on methods to increase response to postal and electronic questionnaires included 513 trials and identified many strategies to increase response to questionnaires (Edwards 2009). Included trials were embedded in surveys, cohort studies and trials, which may explain some of the heterogeneity in effects seen in Edwards' review and reliance on the random-effects model. Unexplained heterogeneity was not a particular problem in this review. Edwards found monetary incentives effective for increasing response to postal questionnaires (Edwards 2009). However, unlike our review, Edwards found that non-monetary incentives were effective for postal and electronic questionnaires. Other strategies found to be effective by Edwards, in agreement with our review, included recorded delivery of questionnaires and shorter questionnaires, although in our review shorter questionnaires need further evaluation. Edwards also found that use of hand-written addresses, stamped return envelopes as opposed to franked return envelopes and first-class outward mailing improved response. Our review found that a 'package' including an enhanced letter incorporating several reminders was effective, but the effectiveness of first-class/priority mail to increase response in randomised trials was unclear.

Booker's narrative review of methods to increase retention in population-based cohort studies was based on only 11 randomised trials and no meta-analysis (Booker 2011). The results suggested that incentives were associated with an increase in retention.

Nakash's systematic review of ways to increase response to postal questionnaires in health care focused on randomised trials of ways to increase response to postal questionnaires in healthcare research on participant populations (Nakash 2006 (2). Fifteen trials were included in this meta-analysis, which found that reminder letters, telephone contact and short questionnaires increased response to postal questionnaires in the context of healthcare research. There was no evidence that incentives were effective. Again, this review was not exclusive to evaluations conducted in randomised trials.

The Edwards review was broad and focused specifically on methods to enhance response to questionnaires and included studies in non-healthcare settings (Edwards 2009). The reviews by Nakash and Booker focused on retention in specific research areas, health care and cohort studies (Booker 2011; Nakash 2006 (2)). Unlike these reviews, our review focused specifically on a range of strategies evaluated within trials. Therefore, it specifically addressed the question of retention of study participants within randomised trials, which was beyond the scope of the other reviews. Application of these results would depend on trial setting, population, disease area, data collection and follow-up procedures. Moreover, we identified additional strategies that may improve trial retention, for example, methodological strategies.

This review is the most comprehensive to date on strategies specifically designed to improve retention in randomised trials. We included seven unpublished trials and 18 other trials not included by Edwards (Edwards 2009).

Authors' conclusions

Implication for methodological research

Trialists may consider including well thought out and adequately powered evaluations of strategies to increase retention in randomised trials. This could include a clear definition of retention strategies and of measures of retention. Trialists conducting future methodology trials can consider incorporating evaluations of strategies to increase retention at the design stage so that power, sample size and funding arrangements are taken into account. Retention trials were often poorly reported without consort diagrams, clear primary outcomes, sample size, sociodemographic composition or power calculations. Considerable time was spent contacting authors for unreported data needed for a robust meta-analysis. Trialists in their reports might consider adhering to the consort guidelines for trial reporting, which would facilitate the synthesis of results in future methodology reviews. There is less research on ways to increase return of participants to trial sites for follow-up and on the effectiveness of strategies to retain trial sites in cluster and individual randomised trials. Research in both areas would be very beneficial to trialists. There is no current system for identifying methodological trials in progress, until a system is set up it may be useful for systematic review authors to incorporate contacting trials units into their search strategy.

Acknowledgements

We would like to thank the following: all authors of included published trials for providing extra unreported data. Principal investigators for data on studies in progress or completed and unpublished: Julia Bailey UCL for data for Bailey 1 and Bailey 2; Graeme MacLennan for data for MacLennan; Stephanie Land for data for Land 2007; the co-ordinators of UK Clinical Trials Units who responded to our survey with information about ongoing or unpublished completed trials; Cara Booker, SPHRU for search strategy information; Angela Young, librarian UCL for assistance with searching databases; and Ian White MRC Biostatistics Unit Cambridge, for helpful comments on the analysis of multiarm trials. Finally, we would like to thank the following: Shaun Treweek, Mike Clarke, Andy Oxman, Karen Robinson and Anne Eisinga for comments on the review protocol and Phil Edwards, Elie Akl, Lisa Maguire, Jean Suvan, Shaun Treweek, Mike Clarke and Karen Robinson for comments on the full review. This project was funded by the MRC Population Health Sciences Research Network.

Data and analyses

Download statistical data

Comparison 1. Addition of incentive vs none: main analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention14 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Addition of monetary incentive33166Risk Ratio (M-H, Fixed, 95% CI)1.18 [1.09, 1.28]
1.2 Addition of offer of monetary incentive/prize draw23613Risk Ratio (M-H, Fixed, 95% CI)1.25 [1.14, 1.38]
1.3 Addition of non-monetary incentive66322Risk Ratio (M-H, Fixed, 95% CI)1.00 [0.98, 1.02]
1.4 Addition of offer of non-monetary incentive21138Risk Ratio (M-H, Fixed, 95% CI)0.99 [0.95, 1.03]
1.5 Addition of offer of monetary donation to charity1815Risk Ratio (M-H, Fixed, 95% CI)1.02 [0.78, 1.32]
Analysis 1.1.

Comparison 1 Addition of incentive vs none: main analysis, Outcome 1 Retention.

Comparison 2. Addition of incentive: sensitivity analysis: quasi-randomised trials removed
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention7 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Addition of monetary incentive21022Risk Ratio (M-H, Fixed, 95% CI)1.31 [1.11, 1.55]
1.2 Addition of non-monetary incentive51594Risk Ratio (M-H, Fixed, 95% CI)1.00 [0.93, 1.08]
Analysis 2.1.

Comparison 2 Addition of incentive: sensitivity analysis: quasi-randomised trials removed, Outcome 1 Retention.

Comparison 3. Addition of incentive: separating research arms of non-factorial trials (three-/four-arm trials)
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention14 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Addition of monetary incentive33066Risk Ratio (M-H, Fixed, 95% CI)1.17 [1.09, 1.27]
1.2 Offer of monetary incentive34224Risk Ratio (M-H, Fixed, 95% CI)1.24 [1.13, 1.37]
1.3 Addition of non-monetary incentive810793Risk Ratio (M-H, Fixed, 95% CI)1.00 [0.98, 1.01]
Analysis 3.1.

Comparison 3 Addition of incentive: separating research arms of non-factorial trials (three-/four-arm trials), Outcome 1 Retention.

Comparison 4. Addition of telephone follow-up vs incentive
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention1 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Telephone survey vs. monetary incentive and questionnaire1700Risk Ratio (M-H, Fixed, 95% CI)1.08 [0.94, 1.24]
Analysis 4.1.

Comparison 4 Addition of telephone follow-up vs incentive, Outcome 1 Retention.

Comparison 5. Addition of monetary incentive to both study arms
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention2902Risk Ratio (M-H, Fixed, 95% CI)1.12 [1.04, 1.22]
1.1 Addition of GBP10 plus offer of GBP10 vs. addition of GBP5 plus offer of GBP51485Risk Ratio (M-H, Fixed, 95% CI)1.16 [1.04, 1.30]
1.2 Addition of GBP20 voucher offer vs. addition of GBP10 voucher offer1417Risk Ratio (M-H, Fixed, 95% CI)1.08 [0.97, 1.21]
Analysis 5.1.

Comparison 5 Addition of monetary incentive to both study arms, Outcome 1 Retention.

Comparison 6. Addition of monetary incentive vs offer of incentive
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention2297Risk Ratio (M-H, Fixed, 95% CI)1.04 [0.91, 1.19]
1.1 Addition of monetary incentive vs. offer of entry into prize draw2297Risk Ratio (M-H, Fixed, 95% CI)1.04 [0.91, 1.19]
Analysis 6.1.

Comparison 6 Addition of monetary incentive vs offer of incentive, Outcome 1 Retention.

Comparison 7. Enhanced letter versus standard letter: main analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention2 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Enhanced letter vs. standard letter22479Risk Ratio (M-H, Fixed, 95% CI)1.01 [0.97, 1.05]
Analysis 7.1.

Comparison 7 Enhanced letter versus standard letter: main analysis, Outcome 1 Retention.

Comparison 8. Communication strategies letter: total design method
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention1 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Total design method for postal questionnaires vs. customary method1226Risk Ratio (M-H, Fixed, 95% CI)1.43 [1.22, 1.67]
Analysis 8.1.

Comparison 8 Communication strategies letter: total design method, Outcome 1 Retention.

Comparison 9. Communication strategies post: main analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention7 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Priority vs. regular post71888Risk Ratio (M-H, Fixed, 95% CI)1.02 [0.95, 1.09]
Analysis 9.1.

Comparison 9 Communication strategies post: main analysis, Outcome 1 Retention.

Comparison 10. Communication strategies: additional reminder vs usual follow-up: main analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention6 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Additional reminder vs. usual follow-up procedures63401Risk Ratio (M-H, Fixed, 95% CI)1.03 [0.99, 1.06]
Analysis 10.1.

Comparison 10 Communication strategies: additional reminder vs usual follow-up: main analysis, Outcome 1 Retention.

Comparison 11. Communication strategies additional reminder to trial site vs usual reminder (ICC 0.054)
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention1 Risk Ratio (Fixed, 95% CI)Subtotals only
1.1 Monthly reminder of upcoming assessments to trial site vs. usual reminders1272Risk Ratio (Fixed, 95% CI)0.96 [0.83, 1.11]
Analysis 11.1.

Comparison 11 Communication strategies additional reminder to trial site vs usual reminder (ICC 0.054), Outcome 1 Retention.

Comparison 12. Communication strategies: questionnaire administered early vs late
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention1 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Early vs. late administration1664Risk Ratio (M-H, Fixed, 95% CI)1.10 [0.96, 1.26]
Analysis 12.1.

Comparison 12 Communication strategies: questionnaire administered early vs late, Outcome 1 Retention.

Comparison 13. Communication strategies: type of reminder: main analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention1 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Recorded delivery vs. telephone reminder1192Risk Ratio (M-H, Fixed, 95% CI)2.08 [1.11, 3.87]
Analysis 13.1.

Comparison 13 Communication strategies: type of reminder: main analysis, Outcome 1 Retention.

Comparison 14. Questionnaire strategies: new vs standard questionnaire: main analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention10 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Short vs. long questionnaire57277Risk Ratio (M-H, Fixed, 95% CI)1.04 [1.00, 1.08]
1.2 Long and clear vs. short and condensed questionnaires1900Risk Ratio (M-H, Fixed, 95% CI)1.01 [0.95, 1.07]
1.3 Question order: condition first vs. generic first questions29435Risk Ratio (M-H, Fixed, 95% CI)1.00 [0.97, 1.02]
1.4 Questionnaire: relevant vs. less relevant to condition23893Risk Ratio (M-H, Fixed, 95% CI)1.07 [1.01, 1.14]
Analysis 14.1.

Comparison 14 Questionnaire strategies: new vs standard questionnaire: main analysis, Outcome 1 Retention.

Comparison 15. Questionnaire strategies: new vs standard questionnaire: sensitivity analysis quasi-randomised trial McColl
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention8 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Short vs. long questionnaire57277Risk Ratio (M-H, Fixed, 95% CI)1.04 [1.00, 1.08]
1.2 Long and clear vs. short and condensed questionnaires1900Risk Ratio (M-H, Fixed, 95% CI)1.01 [0.95, 1.07]
1.3 Questionnaire: relevant vs. less relevant to condition23893Risk Ratio (M-H, Fixed, 95% CI)1.07 [1.01, 1.14]
Analysis 15.1.

Comparison 15 Questionnaire strategies: new vs standard questionnaire: sensitivity analysis quasi-randomised trial McColl, Outcome 1 Retention.

Comparison 16. Behavioural strategies: main analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention2 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Motivation vs. information2273Risk Ratio (M-H, Fixed, 95% CI)1.08 [0.93, 1.24]
Analysis 16.1.

Comparison 16 Behavioural strategies: main analysis, Outcome 1 Retention.

Comparison 17. Case management
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention1 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Case management vs. usual follow-up1703Risk Ratio (M-H, Fixed, 95% CI)1.00 [0.97, 1.04]
Analysis 17.1.

Comparison 17 Case management, Outcome 1 Retention.

Comparison 18. Methodology strategies
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Retention1 Risk Ratio (M-H, Fixed, 95% CI)Subtotals only
1.1 Open vs. blind trial design1538Risk Ratio (M-H, Fixed, 95% CI)1.37 [1.16, 1.63]
Analysis 18.1.

Comparison 18 Methodology strategies, Outcome 1 Retention.

Appendices

Appendix 1. Attrition terms

Cochrane Central Register of Controlled Trials (CENTRAL) and Database of Abstracts of Reviews of Effects (DARE in The Cochrane Library) were searched using attrition terms shown below.

(minimi$ adj2 attrition).ab,ti.

(prevent$ adj2 attrition).ab,ti.

(lessen$ adj2 attrition).ab,ti.

(decreas$ adj2 attrition).ab,ti.

(reduc$ adj2 attrition).ab,ti.

(minimi$ adj2 drop-out).ab,ti.

(prevent$ adj2 drop-out).ab,ti.

(lessen$ adj2 drop-out).ab,ti.

(decreas$ adj2 drop-out).ab,ti.

(reduc$ adj2 drop-out).ab,ti.

(minimi$ adj2 drop-out$).ab,ti.

(prevent$ adj2 drop-out$).ab,ti.

(lessen$ adj2 drop-out$).ab,ti.

(decreas$ adj2 drop-out$).ab,ti.

(reduc$ adj2 drop-out$).ab,ti.

(minimi$ adj2 drop$-out).ab,ti.

(prevent$ adj2 drop$-out).ab,ti.

(lessen$ adj2 drop$-out).ab,ti.

(decreas$ adj2 drop$-out).ab,ti.

(reduc$ adj2 drop$-out).ab,ti.

minimi$ adj2 dropout$).ab,ti.

(prevent$ adj2 dropout$).ab,ti.

(lessen$ adj2 dropout$).ab,ti.

(decreas$ adj2 dropout$).ab,ti.

(reduc$ adj2 dropout$).ab,ti

(strateg$ adj2 drop$-out) .ab,ti.

(strateg$ adj2 dropout$).ab,ti.

 (loss adj2 follow-up).ab,ti.

(lost adj2 follow-up).ab,ti.

(loss adj2 followup).ab,ti.

(lost adj2 followup).ab,ti.

(minimi$ adj2 withdrawal).ab,ti.

(prevent$ adj2 withdrawal).ab,ti.

(lessen$ adj2 withdrawal).ab,ti.

(decreas$ adj2 withdrawal).ab,ti.

(reduc$ adj2 withdrawal).ab,ti.

(minimi$ adj2 withdrawal$).ab,ti.

(prevent$ adj2 withdrawal$).ab,ti.

(lessen$ adj2 withdrawal$).ab,ti.

(decreas$ adj2 withdrawal$).ab,ti.

(reduc$ adj2 withdrawal$).ab,ti.

(strateg$ adj2 attrition).ab,ti.

(strateg$ adj2 drop-out).ab,ti.

(strateg$ adj2 dropout).ab,ti.

(strateg$ adj2 follow-up).ab,ti.

(strateg$ adj2 followup).ab,ti.

(increas$ adj2 retention).ab,ti.

(encourag$ adj2 retention).ab,ti.

(maximi$ adj2 retention).ab,ti.

(promot$ adj2 retention).ab,ti.

(improv$ adj2 retention).ab,ti.

(strateg$ adj2 response$).ab,ti.

(strateg$ adj2 (questionnaire$ adj3 response$)).ab,ti.

(increas$ adj2 (questionnaire$ adj3 response$)).ab,ti.

(encourag$ adj2 (questionnaire$ adj3 response$)).ab,ti.

(maximi$ adj2 (questionnaire$ adj3 response$)).ab,ti.

(promot$ adj2 (questionnaire$ adj3 response$)).ab,ti.

(improv$ adj2 (questionnaire$ adj3 response$)).ab,ti.

(increas$ adj2 response$).ab,ti.

(encourag$ adj2 response$).ab,ti.

(maximi$ adj2 response$).ab,ti.

(promot$ adj2 response$).ab,ti.

(improv$ adj2 response$).ab,ti.

(retention adj2 strateg$).ab,ti.

retention rate$.ab,ti.

(retention adj2 method$).ab,ti.

(retention adj2 technique$).ab,ti.

attrition rate$.ab,ti.

(questionnaire$ adj3 (response$ adj2 method$)).ab,ti.

(questionnaire$ adj3 (response adj2 technique$)).ab,ti.

(questionnaire adj response rate$).ab,ti. (1145)

(difficult$ adj2 (retain$ or retention)).ab,ti.

Participant Dropouts/

The search syntax was adapted as follows for different search interfaces

MEDLINE; EMBASE; PsycINFO via Ovid,

pt- Publication type.

adj2- words within 2 words of each other.

ab- word in abstract.

sh- sub heading.

ti word in title.

/ - Subject heading Medline.

$ - Truncation symbol.

Codes used to de duplicate in Ovid were:

use mesz

use emez

use psyh

Cochrane Central Register of Controlled Trials (CENTRAL) and Database of Abstracts of Reviews of Effects (DARE) via The Cochrane Library

* Truncation symbol

NEAR/2 - words within 2 words of each other.

:kw- keyword

Codes used to de duplicate in CENTRAL were:

"accession number " near pubmed

"accession number " near2 embase

CINAHL (Cumulative Index to Nursing and Allied Health; 1981 to present) via EBSCOHost

MH Major heading (CINAHL via EBSCOHost -)

+- (e.g.Treatment Outcomes+) (CINAHL via EBSCOHost -)

N2 - words within 2 words of each other.

* Truncation symbol.

Campbell Collaboration’s Social, Psychological, Educational and Criminological Trials Register (C2-SPECTR http://geb9101.gse.upenn.edu/

* - Truncation symbol.

Education Resource Information Centre (ERIC) via Dialog datastar.

$ - Truncation symbol

ab- word in abstract.

ti word in title.

 

MeSH Headings

exp Participant Dropouts/: This was used in MEDLINE, only as a subject heading.

In PsycINFOExperimental attrition was used

In CINAHL plusResearch subject retention was used (“research dropouts” - term scope = mechanisms used to keep study participants willing and able to contribute to participate in the study for its duration).

Appendix 2. MEDLINE search strategy

Search strategy for MEDLINE Cochrane Highly Sensitive Search Strategy for identifying randomized trials in MEDLINE: Sensitivity and precision maximising version, 2008 revision Lefebvre 2008; Ovid format.

#1     randomized controlled trial.pt.

#2     controlled clinical trial.pt.

#3     randomized.ab.

#4     placebo.ab.

#5     clinical trials as topic.sh.

#6     randomly.ab.

#7     trial.ti.

#8     1 or 2 or 3 or 4 or 5 or 6 or 7

#9     exp animals/ not humans. sh.

#10 8 not 9

#11 10  AND Attrition terms (Appendix 1)

Appendix 3. EMBASE search strategy

EMBASE search using a sensitivity and specificity maximising search strategy for identifying clinically sound treatment studies Wong 2006a.

#1     random$.tw.

#2     placebo$.ti,ab,sh.

#3     double-blind$.tw.

#4     1 or 2 or 3

#5     4 AND Attrition terms (Appendix 1)

Appendix 4. PsycINFO search strategy

PsycINFO search strategy for identifying high quality studies on treatment: Sensitivity and specificity maximising version. Eady 2008

#1     double-blind.ab,ti.

#2     "random$ assigned.".ab,ti.

#3     control.ab,ti.

#4     1 or 2 or 3

#5      4 AND Attrition terms (Appendix 1)

Appendix 5. CINAHL search strategy

CINAHL search strategy for identifying therapy studies and review articles: Sensitivity and specificity maximising version Wong 2006b

#1  PT Clinical trial

#2  (MH "Treatment Outcomes+")

#3  randomi?ed

#4  1 or 2 or 3

#5 4 AND Attrition terms (Appendix 1)

Appendix 6. C2-SPECTR search strategy

Campbell Collaboration's Social, Psychological, Educational and Criminological Trials Register C2-SPECTR advanced search Non-indexed fields and indexed fields.

Terms used: {retention} OR {attrition} OR {dropout} OR {drop-out} OR {withdrawal} OR {response}.

 

Appendix 7. ERIC search strategy

ERIC search strategy Petrosino 2000

#1 RANDOMI$.TI,AB.

#2 RANDOM$.TI,AB.

#3 (ALLOCAT$ OR ALLOT$ OR ASSIGN$ OR BASIS OR DIVID$ OR ORDER$).TI,AB.

#4 (2 NEAR 3).TI,AB.

#5 RANDOM$.TI,AB. NOT (4 ADJ or1).TI,AB.

#6 ((SINGL$ OR DOUBL$ OR TREBL$ OR TRIPL$) NEAR (BLIND$ OR MASK$)).TI,AB.

#7 ((COMPAR$ OR CONTROL$ OR EXPERIMENT$ OR INTERVENT$ OR THERAP$ OR TREATMENT$) NEAR (GROUP$ OR CLASS$)).TI,AB.

#8 (ALLOCAT$ OR ALLOT$ OR ASSIGN$ OR DIVID$ OR ORDER$).TI,AB.

#9 (7 NEAR 8).TI,AB.

#10 crossover.TI,AB.

#11 (LATIN NEAR SQUARE).TI,AB.

#12 ((CLINIC$ OR CONTROL$) NEAR (TRIAL$ OR STUDY$ OR STUDIES$)).TI,AB.

#13 PLACEBO$

#14 (1 OR 4 OR 5 OR 6 OR 9 OR 10 OR 11 OR 12 OR 13).TI,AB.

#15 Attrition

#16 (attrition ADJ research ADJ studies). TI,AB.

#17 14 AND 16

#18 17 AND Attrition terms (Appendix 1)

Appendix 8. Reference lists of reviews and other publications searched

  1. Torgerson G. Should we pay the patient? Review of financial incentives to enhance patient compliance. BMJ 1997;315.

  2. Jeffery M, Hickey BE, Hider PN. Follow-up strategies for patients treated for non-metastatic colorectal cancer. Cochrane Database of Systematic Reviews 2007 Art No : CD002200 DOI: 10 1002/14651858 CD002200 pub2 2011.

  3. Rudy EB, Estok PJ, Kerr ME, Menzel L. Research incentives: money versus gifts. Nursing Research 1994;43:253.

  4. Bruzzese JM, Gallagher R, McCann-Doyle S, Reiss PT, Wijetunga NA. Effective methods to improve recruitment and retention in school-based substance use prevention studies. Journal of School Health 2009;79:400-7.

  5. Rojas MP, Telaro E, Russo A, Moschetti I, Coe L, Fossati R et al. Follow-up strategies for women treated for early breast cancer. Cochrane Database of Systematic Reviews, 2000, Art. No.: CD001768. DOI:10.1002/14651858.CD001768.pub2.

  6. Froelicher ES, Miller NH, Buzaitis A, Pfenninger P, Misuraco A, Jordan S et al. The Enhancing Recovery in Coronary Heart Disease Trial (ENRICHD): strategies and techniques for enhancing retention of patients with acute myocardial infarction and depression or social isolation. Journal of Cardiopulmonary Rehabilitation 2003;23:269-80.

  7. Sarre G, Dyas J, Clarke A, Keen C. Planning Recruitment Options: Strategies for Primary Care Research. 2009.

  8. Heo M, Papademetriou E, Meyers BS. Design characteristics that influence attrition in geriatric antidepressant trials: meta-analysis. International Journal of Geriatric Psychiatry 2009;24:990-1001.

  9. Fabricatore AN, Wadden TA, Moore RH, Butryn ML, Gravallese EA, Erondu NE et al. Attrition from randomized controlled trials of pharmacological weight loss agents: a systematic review and analysis. Obesity Reviews 2009;10:333-41.

  10. Northouse LL, Rosset T, Phillips L, Mood D, Schafenacker A, Kershaw T. Research with families facing cancer: the challenges of accrual and retention. Research in Nursing & Health 2006;29:199-211.

  11. Mc Daid C, Hodges Z, Fayter D, Stirk L, Eastwood A. Increasing participation of cancer patients in randomised controlled trials: a systematic review. Trials 2006;7:16.

  12. McFarlane J: Strategies for successful recruitment and retention of abused women for longitudinal studies. Issues in Mental Health Nursing 2007;28:883-97.

  13. Burns D, Soward ACM, Skelly AH, Leeman J, Carlson J. Effective recruitment and retention strategies for older members of rural minorities. The Diabetes Educator 2008;34:1045-52.

  14. Davis LL, Broome ME, Cox R. Maximizing retention in community-based clinical trials. Journal of Nursing Scholarship 2002;34:47-53.

  15. Edwards P, Roberts I, Sandercock P, Frost C. Follow-up by mail in clinical trials: does questionnaire length matter? Control Clinical Trials 2004;25:31-52.

  16.  Edwards P, Cooper R, Roberts I, Frost C. Meta-analysis of randomised trials of monetary incentives and response to mailed questionnaires. Journal of Epidemiology and Community Health 2005;59:987-99.

  17. Edwards PJ, Roberts IG, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I et al. Methods to increase response rates to postal and electronic questionnaires. Cochrane Database of Systematic Reviews 2009, Issue 3 Art No: MR000008 2009.

  18. Nakash R, Hutton J, Jorstad-Stein E, Gates S, Lamb S. Maximising response to postal questionnaires - a systematic review of randomised trials in health research. BMC Medical Research Methodology 2006;6:5.

  19. Robinson KA, Dennison CR, Wayman DM, Pronovost PJ, Needham DM. Systematic review identifies number of strategies important for retaining study participants. Journal of Clinical Epidemiology 2007;60:757.

  20. Schellings R, Kessels AG, ter Riet G, Knottnerus JA, Sturmans F. Randomized consent designs in randomized controlled trials: systematic literature search. Contemporary Clinical Trials 2006;27:320-32.

  21. Warner KE, Murt HA. Economic incentives for health. Annual Review of Public Health 1984;5:107-33.

  22. Wiemann CM, Chacko MR, Tucker JD, Velasquez MM, Smith PB, DiClemente RJ et al. Enhancing recruitment and retention of minority young women in community-based clinical research. Journal of Pediatric and Adolescent Gynecology 2005;18:403-7.

  23. Yabroff KR, Kerner JF, Mandelblatt JS. Effectiveness of interventions to improve follow-up after abnormal cervical cancer screening. Preventive Medicine 2000;31:429-39.

  24. van der Bij AK, Laurant MGH, Wensing M. Effectiveness of physical activity interventions for older adults: a review. American Journal of Preventive Medicine 2002;22:120-33.

  25. Yancey AK, Ortega AN, Kumanyika SK. Effective recruitment and retention of minority research participants. Annual Review of Public Health 2006;27:1-28.

  26. Scott P, Edwards P. Personally addressed hand-signed letters increase questionnaire response: a meta-analysis of randomised controlled trials. BMC Health Services Research 2006;6:111.

Appendix 9. Characteristics of host trials

Clinical area main trial Condition Participants Setting Attrition Study

Treatment of dependence

 

AlcoholAdults scoring +5 on AUDIT-C, mean age 37 years in an online trial comparing interactive computer intervention plus website information vs. website information for modifying alcohol intake Murray 2007Community: online Khadjesari 2011 1abc
 AlcoholAdults scoring +5 on AUDIT-C mean age 37 years in an online trial comparing interactive computer intervention plus website information vs. website information for modifying alcohol intake Murray 2007Community: online 

Khadjesari 2011 2

 

 AlcoholAdults scoring +5 on AUDIT-C mean age 37 years in an online trial comparing interactive computer intervention plus website information vs. website information for modifying alcohol intake Murray 2007Community: online McCambridge 2011 1
 AlcoholAdults scoring +5 on AUDIT-C mean age 37 years in an online trial comparing interactive computer intervention plus website information vs. website information for modifying alcohol intake Murray 2007Community: online McCambridge 2011 2
 

Smoking

 

Adult smokers aged 38-77 years in a smoking cessation trial of public education through media and community wide events, healthcare providers work sites and other organisations vs. no intervention Gail 1992

USA community

 

Bauer 2004ab

 

 

Smoking

 

Adult smokers mean age 36.7 years in a trial of nicotine gum vs. placebo gum. Smokers for > 1 year Hughes 1984

USA community

 

Hughes 1989

 

 

Smoking

 

Adult smokers willing to quit aged > 16 years in a  trial  comparing Txt2stop motivational messages and behaviour change support vs. text messages unrelated to quitting Free 2011

UK community

 

Severi 2011 1

 

 SmokingAdult smokers willing to quit aged > 16 years in a  trial comparing Txt2stop motivational messages and behaviour change support vs. text messages unrelated to quitting Free 2011UK community Severi 2011 2
Treatment of injuryNeckMINT trial: adults with whiplash injury 18-87 years in a 2 x 2 cluster randomised trial comparing whiplash book vs. usual advice. Individuals randomised to physiotherapy vs. single advice session reinforcing advice given Lamb 2007UK  hospital trusts Gates 2009
 AnkleCast trial: adults aged 16-57 years with acute severe ankle sprain in a trial comparing tubular bandage vs. below knee cast vs. Aircast® ankle brace vs. Bledsoe® boot Cooke 2009UK accident and emergency departments

Nakash 2007*

 

 HeadAdults with head injury aged > 16 years in trial of 48-hour infusion of methylprednisolone vs. placebo CRASH Trial collaborators 2004UK hospital intensive care units Edwards 2001*
 HeadAdults with head injury > 16 years CRASH trial: 48-hour infusion of methylprednisolone vs. placebo CRASH Trial collaborators 2004Czech republic hospital intensive care units Svoboda 2001*
Treatment of diseaseBreast cancerWomen with ductal carcinoma in situ aged > 49 years in a trial comparing anastrozole vs. tamoxifen (unpublished)USA, Canada, Puerto Rico hospitals Land 2007
 StrokeAcute stroke patients 50-80 years in an international stroke trial of heparin 125,000 IU twice daily + aspirin 300 mg daily vs. heparin 125,000 IU twice vs. heparin 5000 IU twice daily + aspirin 300 mg daily, heparin 5000 IU twice daily vs. aspirin 300 mg daily vs. no heparin or aspirin International Stroke Trial Group 1997UK hospital Dorman 1997
 Ventricular fibrillation, ventricular tachycardiaAdults cardioverted from ventricular tachycardia or resuscitated from ventricular fibrillation aged 54-76 years participating in the AVID trial comparing an implanted cardioverter defibrillator vs. antiarrhythmic drugs AVID investigators 1997USA hospital Renfroe 2002a-d
 EpilepsyAdults with epilepsy mean 38.3 years in the SANAD trial. Arm a: carbamazepine vs. gabapentin vs. lamotrigine vs. oxcarbazepine vs. topiramate. Arm b: valproate vs. lamotrigine (LTG) vs. vs topiramate (TPM) Marson 2007 (2)UK hospital outpatient departments Marson 2007
 Back painAdults with low back pain aged 18-65 years in a trial comparing exercise manipulation vs. exercise plus manipulation UK BEAM 2004UK primary care Letley 2000
 Back painAdults aged 18-65 years in a trial comparing yoga vs. usual care Tilbrook 2011UK primary care Man 2011
ScreeningProstate, lung, ovarian, colorectal cancerAdults aged 55-74 years in PLCO trial comparing PSA and CA125 at baseline, and annually for 5 years. Digital rectal examination, transvaginal ultrasound and chest x-ray at baseline and 5 years vs. usual follow-up Prorok 2000USA trial sites

Subar 2001,

Ford 2006 (African American men aged ≥ 55 years only from PLCO)

 CervicalWomen with low-grade abnormal cervical smear aged 20-59 years in the TOMBOLA trial: Colposcopy vs. 6-monthly smears TOMBOLA 2009a TOMBOLA 2009bUK Sharp 2006a-h
 Postnatal depressionWomen childbearing aged > 18 years < 2 weeks' postpartum at high risk of postnatal depression in a trial comparing proactive individualised telephone-based peer support vs. standard postpartum care Dennis 2009Canada community Kenton 2007a-d
Prevention

Fracture

 

Adults with history of osteoporotic fracture > 70 years in the RECORD trial: oral calcium + vitamin D vs. oral calcium vs. vitamin D vs. placebo RECORD 2007UK hospital MacLennan*
 FractureAdults with history of osteoporotic fracture aged > 70 years in the RECORD trial: oral calcium + vitamin D vs. oral calcium  vs. vitamin D vs. placebo RECORD 2007UK hospital Avenell 2004
 FractureWomen  with hip fracture risk factors aged > 70 years in a fracture prevention trial of calcium 1000 mg plus vitamin D3 800 IU plus information sheet on dietary calcium intake and falls prevention vs. information sheet Porterhouse 2005UK primary care Cockayne 2005
 

Migraine

 

Adults history of 2 migraine attacks aged 18-65 years with migraine randomised to true diet vs. sham diet (unpublished)UK community Ashby 2011
 Lung cancerAdults exposed to smoking and asbestos aged > 45 years in the CARET trial 2 x 2: beta-carotene + retinol daily vs. beta-carotene vs. retinol vs. placebo Omenn 2006USA trial sites

Bowen 2000abc

 

 Breast cancerWomen with 50% of breast volume dysplasia > 30 years in Canadian diet and cancer prevention trial. Counselling and individualised dietary prescription vs. taught principals of a healthy diet not counselled to change fat content Boyd 2002Canada Hospital clinic Sutherland 1996
Clinical managementAsthmaAdult with asthma > 70 years in COGENT trial: computerised decision support guidelines for asthma vs. angina care Eccles 2002UK primary care McColl 2003
 Asthma and diabetesAdults with asthma mean age 47 years. Study template for diabetes vs. study template for asthma Tai 1999UK primary care Tai 1997
 

Angina

 

Adult with asthma aged > 70 years in the COGENT trial: computerised decision support guidelines for asthma vs angina care Eccles 2002UK primary care McColl 2003
 OrthopaedicsAdults with non-surgical musculoskeletal condition ≥ 18 years in OMENS trial: orthopaedic patients management from single musculoskeletal medicine physician vs. orthopaedic surgeon-led management  Leigh Brown 1997UK Hospital othhopaedic outpatient department Leigh Brown 1997
Other areasExerciseWomen sedentary 50-70 years. SWEAT2 trial: moderate walking programme vs. swimming programme Cox 2008Australia community Cox 2008
 ParentingAdults referred for parenting mean age 29 years. Parent-child interactive therapy vs. standard didactic parenting condition Chaffin 2009USA community Chaffin 2009
 Weight managementAdults with BMI > 25 aged > 18 years. Internet-based tailored weight management materials vs. Internet-based non-tailored user navigated weight management materials Rothert 2006USA community Couper 2007
 Effect of antibiotics on neonatal outcomesWomen < 37 weeks' gestation in ORACLE 1 + 2 trial: 2 x 2 factorial: co-amoxiclav + erythromycin vs. co-amoxiclav vs. erythromycin vs. placebo; 4 times daily x 10 days or until birth Kenyon 2005UK secondary care/community Kenyon 2005
 

Health promotion

 

People aged 16-20 years in the sex unzipped pilot feasibility trial: interactive intervention website vs. information only website (main trial unpublished)

UK on line

 

Bailey 1*

 

 Health promotionPeople aged 16-20 years in the sex unzipped pilot feasibility trial: interactive intervention website vs. information only website (main trial unpublished)UK on line Bailey 2*

* unpublished; ‡ unpublished abstract; † report; ⁑published abstract; appendix.

AUDIT: Alcohol Use Disorders Identification Test; AVID: Antiarrhythmics Versus Implantable Defibrillators; BMI: body mass index; CARET: Carotene and Retinol Efficacy Trial; COGENT: COlorectal cancer GENeTics; CRASH: Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage; IU: international unit; OMENS: ; ORACLE: ; PLCO: Prostate, Lung, Colorectal, Ovarian; PSA: prostate-specific antigen; RECORD: Randomised Evaluation of Calcium Or vitamin D; SANAD: Standard And New Antiepileptic Drugs; SWEAT: Sedentary Women Exercise Adherence Trial; TOMBOLA: Trial Of Management of Borderline and Other Low-grade Abnormal smears.

Contributions of authors

VB wrote the protocol for the review with comments from JT, GR, SS, SM, SH and IN. JT and VB designed the searches with comments from SH. VB conducted the searches and screened all abstracts and full papers of potentially eligible trials. VB and GR screened potentially eligible trial papers. SS acted as a third reviewer. Data extraction was conducted by VB and checked by JT. JT designed the analysis plan with VB. VB conducted the analysis with advice on interpretation of results from JT, SS, IN and GR. VB wrote the first draft of the review with comments from all authors.

Declarations of interest

None known.

Sources of support

Internal sources

  • None, Not specified.

External sources

  • None, Not specified.

Differences between protocol and review

We grouped trials according to: the type of strategy used, whether the strategy was compared with usual follow-up or other strategies. The diversity of trials and interventions identified in the review were not anticipated. Therefore, the prespecified analyses were not appropriate. Instead, new subgroups were defined prior to analyses. We had planned to include trials that were targeted at treatment or follow-up compliance. We have only included trials that targeted follow-up compliance as the strategies used are transferable to other trials. We had planned to assess whether retention was immediate or longer term (e.g. if a response to a questionnaire was expected immediately or at time points in the future). Time points were poorly reported, where these were reported we used data for this analysis for the primary outcome time point, there was insufficient reporting of this variable to group other time points for further analysis. We had also planned to group participant or management-focused strategies. Only one unpublished trial (Land) evaluated a management-focused strategy to reduce attrition.

As treatment compliance was not a focus of this review, search strategies with the terms 'compliance' were removed for the 2009 to 2012 updates. Most untruncated 'response' search terms were removed because hits relating to 'response' were captured by the search term 'response*'. To avoid references to treatment response, the search term 'questionnaire' was added to 'response' in all remaining search terms with 'response' or 'response*' to make the search specific to questionnaire response. Search updates (2009 to 2012) for EMBASE, MEDLINE and PsycINFO were de duplicated in Ovid. MEDLINE and EMBASE records were excluded for search updates in CENTRAL. C2 Spectre and ERIC searches were not updated from 1 May 2009, as C2-SPECTR geb9101.gse.upenn.edu/: was not accessible and the search platform for ERIC changed from Datastarweb to Proquest in December 2011. The latter limits searches to 10 lines of text.

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Ashby 2011

Methods2-arm randomised trial
DataAdults aged 18-65 years who provided email or mobile telephone contact details for receiving electronic reminders for follow-up in a migraine prevention trial
Comparisons

1. Electronic reminder: either SMS text message, email message, or both sent after the 4-week follow-up study questionnaire sent

2. No electronic reminder sent

Outcomes

Primary: questionnaire response rate defined as proportion of questionnaires returned by participants at final analysis at 40 days

Secondary: time to response

NotesRetention trial embedded in a randomised trial evaluating the effectiveness of food elimination diet based on the ELISA test for food sensitivity for prevention of migraine. Primary outcome for the migraine prevention trial (host trial): change in the number of headache days over 12 weeks using the migraine disability assessment questionnaire (MIDAS). Retention trial identified through mail out to UK clinical trials units
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthors response "an independent data manager at the trials unit was responsible for generating the allocation sequence and assigning participants"
Adequate sequence generation?Unclear"Randomly generated numbers used to list all participants by ID [identification] number who had provided a mobile phone number and/or and email address. The first half of listed participants were allocated to the intervention group the remaining participants were allocated to the control group"
Blinding?UnclearNo reference to blinding of either participants or outcome assessors in the study report
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement

Avenell 2004

MethodsRandomised trial
DataAdults aged ≥ 70 years with a low trauma osteoporotic fracture in the past 10 years recruited in 1 centre of the RECORD trial
Comparisons

1. An open version of the RECORD trial otherwise identical in design

2. RECORD trial, a randomised double-blind placebo-controlled factorial design of oral calcium 1 g daily and or vitamin D 800 IU/20 µg supplementation

Outcomes

Proportion of eligible participants recruited

Proportion remaining in the trial at 1 year

Proportion compliant on pill counts at 8 months

NotesOpen version of the Randomised Evaluation of Calcium or Vitamin D (RECORD) Trial treated as the intervention group in the analysis. Proportion retained at 4, 8 and 12 months were reported. Primary outcome for the randomised double-blind placebo-controlled version of the RECORD trial was all new low-energy fractures
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Nurse used a pre programed laptop computer"
Adequate sequence generation?Yes"A pre programed laptop computer to generate random allocation"
Blinding?YesDouble-blind randomised trial design compared with an open trial design. For the double-blind randomised trial "allocation remained concealed until the final analyses". "All outcomes were reported or verified by people who were masked to the allocation scheme"
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement

Bailey 1

MethodsRandomised trial embedded in the sex unzipped website feasibility trial
DataUK English speaking people aged 16-20 years
Comparisons

1. Offer of GBP20 voucher

2. Offer of GBP10 voucher

OutcomesRetention of participants at 3-month follow-up, i.e. completion of sexual health survey
Notes

Retention trial identified through personal correspondence with the author

Sexunzipped website evaluated in an online trial

Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthors response "the trial statisticians generated the randomisation sequence with participants identified by ID [identification] number only and the trial manager implemented it manually"
Adequate sequence generation?YesAuthors response "participants were randomised after recruitment but before follow-up to a GBP10 or GBP20 incentive. Randomisation to increased incentive was through simple permutation of the list of remaining recruits"
Blinding?UnclearAuthors response "allocation sequences were generated without participants' knowledge". "For those allocated to the increased amount of GBP20, this was revealed in a 3 month follow-up email. Those allocated to GBP10 were not aware that others were offered GBP20 (unless friends had enrolled and had discussed the study). Since the trial recruited participants online from all over the UK, this will have reduced the chance of bias due to contamination"
Free of selective outcome reporting?YesReports the primary outcome

Bailey 2

MethodsRandomised trial
DataUK English speaking people aged 16-20 years
Comparisons

1. Offer of GBP20 voucher: GBP10 in advance and GBP10 on receipt of questionnaire and chlamydia kit

2. Offer of GBP10 voucher: GBP5 in advance and GBP5 on receipt of questionnaire and chlamydia kit

OutcomesRetention of participants at 3-month follow-up, i.e. completion of sexual health survey and return of chlamydia kit
Notes

Study identified through personal correspondence with author

Sex unzipped website evaluated in an online trial

Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthors response "the trial statisticians generated the randomisation sequence (with participants identified by ID [identification] number only), and the trial manager implemented it manually"
Adequate sequence generation?YesAuthors response "participants were randomised after recruitment but before follow-up to a GBP10 or GBP20 incentive. Randomisation to increased incentive was through simple permutation of the list of remaining recruits"
Blinding?UnclearAuthors response "allocation sequences were generated without participants' knowledge". "For those allocated to the increased amount of GBP20, this was revealed in a 3 month follow-up email. Those allocated to GBP10 were not aware that others were offered GBP20 (unless friends had enrolled and had discussed the study). Since the trial recruited participants online from all over the UK, this will have reduced the chance of bias due to contamination"
Free of selective outcome reporting?YesReports the primary outcome

Bauer 2004a

Methods3-arm randomised trial (first incentive vs. no incentive)
DataPilot study of 15 randomly selected participants from each of 20 communities participating in COMMIT trial
Comparisons

Enclosed with mouthwash swish collection kits sent to participants "subjects were further randomised to receive" either:

  1. USD10 cheque told to keep the cheque whether or not they participated, sent with covering letter and prepaid envelope

  2. No incentive, kit sent with covering letter and prepaid envelope

All sent 2 weeks after an advance letter with a professionally rendered brochure

OutcomesPercentage of mouthwash kits returned reported. No time point given
NotesStudy embedded in the cluster randomised Community Intervention Trial for Smoking Cessation (COMMIT) trial. Primary outcome: quit rate among heavy smokers. Several attempts to contact authors regarding allocation sequence. Data extracted from Edwards Cochrane review on response to postal questionnaires
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearNo reply from author
Adequate sequence generation?Unclear"Subjects were further randomised to receive an incentive of....."
Blinding?UnclearInsufficient information to make a judgement about this, no reply from author
Free of selective outcome reporting?UnclearNo reply from author
Other sources of bias?UnclearInsufficient information to make a judgement

Bauer 2004ab

Methods3-arm randomised trial (combined incentive vs. no incentive)
DataPilot study of 15 randomly selected participants from each of 20 communities participating in COMMIT trial
Comparisons

Enclosed with mouthwash swish collection kits sent to participants "subjects were further randomised to receive" either:

  1. USD10 cheque told to keep the cheque whether or not they participated sent with covering letter and prepaid envelope or

  2. USD2 cheque with covering letter and prepaid envelope or

  3. No incentive, kit sent with covering letter and prepaid envelope

All sent 2 weeks after an advance letter with a professionally rendered brochure

OutcomesPercentage of mouthwash kits returned reported. No time point given
NotesStudy embedded in the cluster randomised Community Intervention Trial for Smoking Cessation (COMMIT) trial. Primary outcome: quit rate among heavy smokers. Several attempts to contact authors regarding allocation sequence. Data extracted from Edwards Cochrane review on response to postal questionnaires
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearNo reply from author
Adequate sequence generation?Unclear"Subjects were further randomised to receive an incentive of....."
Blinding?UnclearInsufficient information to make a judgement about this no reply from author
Free of selective outcome reporting?UnclearNo reply from author
Other sources of bias?UnclearInsufficient information to make a judgement

Bauer 2004b

Methods3-arm randomised trial (second incentive vs. no incentive)
DataPilot study of 15 randomly selected participants from each of 20 communities participating in COMMIT trial
Comparisons

Enclosed with mouthwash swish collection kits sent to participants, "subjects were further randomised to receive" either:

  1. USD2 check with covering letter and prepaid envelope

  2. No incentive, kit sent with covering letter and prepaid envelope

All sent 2 weeks after an advance letter with a professionally rendered brochure

OutcomesPercentage of mouthwash kits returned reported. No time point given
NotesStudy embedded in the cluster randomised Community Intervention Trial for Smoking Cessation (COMMIT) trial. Primary outcome: quit rate among heavy smokers. Several attempts to contact authors regarding allocation sequence. Data extracted from Edwards Cochrane review on response to postal questionnaires
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearNo reply from author
Adequate sequence generation?Unclear"Subjects were further randomised to receive an incentive of....."
Blinding?UnclearInsufficient information to make a judgement about this
Free of selective outcome reporting?UnclearNo reply from author
Other sources of bias?UnclearInsufficient information to make a judgement

Bowen 2000a

MethodsRandomised 2 x 2 factorial trial (first incentive vs. no incentive)
DataAdults aged 50-69 years recruited from 2 CARET trial sites, participating in the Participant Retention Item Distribution Evaluation (PRIDE) trial
Comparisons
  1. Certificate of appreciation preprinted on an 8.5 x 11 inch (21.59 x 27.94 cm)bond off-white paper with gold trim and bold, black lettering. The certificate read "[participant's name] in recognition of your contribution to an important national study for the prevention of lung cancer, CARET, CancerPrevention Study, sponsored by The National Cancer Institute". The participant's name was computer printed in an attractive font on the certificate. Each certificate had the signatures of the Co-ordinating Center's principal investigator, study centre investigator and CARET's project officer from the National Cancer Institute, given during a visit for randomisation or follow-up

  2. No incentive

OutcomesPrimary: time of first inactivation (stop taking vitamins or placebos) during the 2-year follow-up period of PRIDE
NotesPrimary disease outcome for the Carotene and Retinol Efficacy Trial (CARET): lung cancer
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearAuthor response "the Coordination Center supplied one date-labelled envelope for each day in the enrolment period to the two participating study centres. The envelopes contained the randomisation assignment for the given day. At the end of each day during the randomisation period, study centre staff opened the envelope containing the intervention assignment for the next day. Study centre staff members were otherwise blinded to the allocation sequence" 
Adequate sequence generation?YesAuthor response "we used a block randomisation approach (stratified by study centre) with a 1:1:1:1 intervention arm allocation ratio, where the randomisation unit was the date of enrolment". Note: treated this as quasi-randomised in the analysis
Blinding?YesAuthor response "with IRB [Institutional Review Board] approval, the study was conducted without participants' knowledge of this research. Thus, participants were blinded to their own intervention only in the sense that they were unaware they were randomised to receive particular item(s)"
Free of selective outcome reporting?YesCumulative incidence of individuals who became inactive during 2-year follow-up reported
Other sources of bias?UnclearInsufficient information to make a judgement

Bowen 2000abc

MethodsRandomised 2 x 2 factorial trial (combined incentive vs. no incentive)
DataAdults aged 50-69 years recruited from 2 CARET trial sites, participating in the Participant Retention Item Distribution Evaluation (PRIDE) trial
Comparisons
  1. Certificate of appreciation preprinted on an 8.5 x 11 inch (21.59 x 27.94 cm)bond off-white paper with gold trim and bold, black lettering. The certificate read '[participant’s name] in recognition of your contribution to an important national study for the prevention of lung cancer, CARET, CancerPrevention Study, sponsored by The National Cancer Institute'. The participant's name was computer printed in an attractive font on the certificate. Each certificate had the signatures of the Co-ordinating Center's principal investigator, study centre investigator, and CARET's project officer from the National Cancer Institute, given during a visit for randomisation or follow-up (arm a)

  2. Lapel pin 1 inch (2.5 cm) in size and designed in cloisonne. Choice between a pin with 6 colours with inscription 'CARET NCI prevention study' and an orange carrot in the middle of the pin or a pin with 5 colours with inscription 'PARTICIPNAT CARET Cancer Prevention Study' and 'Sponsored by NCI' and given during a visit for randomisation or follow-up (arm b)

  3. Certificate of appreciation (details as before) and lapel pin (details as before) and given during a visit for randomisation or follow-up (arm c)

  4. No incentive

OutcomesPrimary: time of first inactivation (stop taking vitamins or placebos) during the 2-year follow-up period of PRIDE
NotesPrimary disease outcome for the Carotene and Retinol Efficacy Trial (CARET): lung cancer
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearAuthor response "the Coordination Center supplied one date-labelled envelope for each day in the enrolment period to the two participating study centres. The envelopes contained the randomisation assignment for the given day. At the end of each day during the randomisation period, study centre staff opened the envelope containing the intervention assignment for the next day. Study centre staff members were otherwise blinded to the allocation sequence"
Adequate sequence generation?YesAuthor response "we used a block randomisation approach (stratified by study centre) with a 1:1:1:1 intervention arm allocation ratio, where the randomisation unit was the date of enrolment"
Blinding?YesAuthor response "with IRB [Institutional Review Board] approval, the study was conducted without participants’ knowledge of this research. Thus, participants were blinded to their own intervention only in the sense that they were unaware they were randomised to receive particular item(s)"
Free of selective outcome reporting?YesCumulative incidence of individuals who became inactive during 2 year follow-up reported
Other sources of bias?UnclearInsufficient information to make a judgement

Bowen 2000b

MethodsRandomised 2 x 2 factorial trial (second incentive vs. no incentive)
DataAdults aged 50-69 years recruited from 2 CARET trial sites, participating in the Participant Retention Item Distribution Evaluation (PRIDE) trial
Comparisons
  1. Lapel pin 1 inch (2.5 cm) in size and designed in cloisonne. Choice between a pin with 6 colours with inscription 'CARET NCI prevention study' and an orange carrot in the middle of the pin or a pin with 5 colours with inscription 'PARTICIPNAT CARET Cancer Prevention Study' and "Sponsored by NCI" and given during a visit for randomisation or follow-up

  2. No incentive

OutcomesPrimary: time of first inactivation (stop taking vitamins or placebos) during the 2-year follow-up period of PRIDE
NotesPrimary disease outcome for the Carotene and Retinol Efficacy Trial (CARET): lung cancer
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearAuthor response "the Coordination Center supplied one date-labelled envelope for each day in the enrolment period to the two participating study centres. The envelopes contained the randomisation assignment for the given day. At the end of each day during the randomisation period, study centre staff opened the envelop containing the intervention assignment for the next day. Study centre staff members were otherwise blinded to the allocation sequence"
Adequate sequence generation?YesAuthor response "we used a block randomisation approach (stratified by study centre) with a 1:1:1:1 intervention arm allocation ratio, where the randomisation unit was the date of enrolment"
Blinding?YesAuthor response "with IRB [Institutional Review Board] approval, the study was conducted without participants’ knowledge of this research. Thus, participants were blinded to their own intervention only in the sense that they were unaware they were randomised to receive particular item(s)"
Free of selective outcome reporting?YesCumulative incidence of individuals who became inactive during 2-year follow-up reported
Other sources of bias?UnclearInsufficient information to make a judgement

Bowen 2000c

MethodsRandomised 2 x 2 factorial trial (third incentive vs. no incentive)
DataAdults aged 50-69 years recruited from 2 CARET trial sites, participating in the Participant Retention Item Distribution Evaluation (PRIDE) trial
Comparisons
  1. Certificate of appreciation (details as before) and lapel pin (details as before) and given during a visit for randomisation or follow-up

  2. No incentive

OutcomesPrimary: time of first inactivation (stop taking vitamins or placebos) during the 2-year follow-up period of PRIDE
NotesPrimary disease outcome for the Carotene and Retinol Efficacy Trial (CARET): lung cancer
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearAuthor response "the Coordination Center supplied one date-labelled envelope for each day in the enrolment period to the two participating study centres. The envelopes contained the randomisation assignment for the given day. At the end of each day during the randomisation period, study centre staff opened the envelop containing the intervention assignment for the next day. Study centre staff members were otherwise blinded to the allocation sequence"
Adequate sequence generation?YesAuthor response "we used a block randomisation approach (stratified by study centre) with a 1:1:1:1 intervention arm allocation ratio, where the randomisation unit was the date of enrolment"
Blinding?YesAuthor response "with IRB [Institutional Review Board] approval, the study was conducted without participants’ knowledge of this research. Thus, participants were blinded to their own intervention only in the sense that they were unaware they were randomised to receive particular item(s)"
Free of selective outcome reporting?YesCumulative incidence of individuals who became inactive during 2-year follow-up reported
Other sources of bias?UnclearInsufficient information to make a judgement

Chaffin 2009

Methods2-arm randomised trial
DataAll parents referred for parenting services at a small inter-city non-profit community agency operating a parenting programme
Comparisons

1. Initial preparenting orientation condition self motivation group

2. Initial preparenting orientation condition standard informational group

OutcomesDropout from the parenting group at 12 weeks
NotesA second randomisation was performed after completion of the orientation programme to parent child interactive therapy vs. standard didactic parenting condition. Dropout recorded at 2-week intervals up to 12 weeks
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?No"Unblinded randomisation list"
Adequate sequence generation?Yes"Computer generated randomisation list"
Blinding?YesAuthor response "they (parents) were informed only in general terms that we were interested in which types of services helped". Participant interviews were conducted by computer. Blinding of personnel: observational parent-child interaction coding was done by personnel who were not informed about intervention condition assignment
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?No"It is possible that therapist effects may have played a role in the outcomes"

Cockayne 2005

Methods2-arm randomised trial
DataWomen aged ≥ 70 years randomised at 1 centre for a fracture prevention trial due to receive their final follow-up questionnaire in March 2004
Comparisons

1. Final follow-up questionnaire additional question offering results of the trial delivered by post

2. Final follow-up questionnaire no offer of study results delivered by post

Both groups received a personalised cover letter showing university sponsorship, along with a business reply envelope. Non-responders within 3 weeks were sent up to 2 reminder letters, questionnaires and business reply envelopes, 3 and 6 weeks after the initial mailing          

OutcomesReturn of final follow-up questionnaire by participants. Time point not specified
Notes

Authors contacted to confirm numbers randomised to each arm

The Fracture Prevention Trial: calcium 1000 mg plus vitamin D3 800 IU plus information sheet on dietary calcium intake and falls prevention vs. information sheet. Primary outcome for the fracture prevention trial: all clinical fractures excluding those of the digits, rib, face and skull

Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"An independent researcher from the York trials unit randomised eligible women"
Adequate sequence generation?Yes"Randomised eligible women in a 3:1 ratio by computer"
Blinding?Unclear"Administration of the questionnaire was not blind to the group allocation"
Free of selective outcome reporting?YesAll outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement

Couper 2007

Methods2-arm randomised trial (incentive plus postal questionnaire vs. telephone survey)
DataAdults ≥ 18 years, BMI of ≥ 25 participating in an Internet-based weight management trial who did not respond to the 12-month questionnaire
Comparisons

1. Telephone call and survey by trained interviewers. Repeated up to 15 times. Attempts made on various days and at various times of the day

2. Postal questionnaire with return address and covering letter signed by directors plus a USD5 bill

OutcomesQuestionnaire response. No time point given
NotesInternet-based weight management trial compared Internet-based tailored weight management materials with Internet-based non-tailored user navigated weight management materials. Primary outcome for the Internet-based trial: percentage of baseline weight lost
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearNot described
Adequate sequence generation?UnclearAuthor response "randomly assigned"
Blinding?UnclearNot described
Free of selective outcome reporting?UnclearUnclear at the outset what is to be reported
Other sources of bias?UnclearInsufficient information to make a judgement

Cox 2008

Methods2-arm randomised trial
DataAll sedentary women aged 50-70 years participating in the SWEAT2 trial
Comparisons

1. 12 work sheets with strategies for goal setting, time management and overcoming barriers to attendance, mini workshops, received worksheets to complete at home - after 6 months received newsletters only. Intervention delivered by a trained facilitator, before an exercise session

2. Information sheets about programme requirements exercise techniques, safety plus 9 newsletters and a report on fitness at 6 months delivered by principle investigator plus 9 newsletters

OutcomesProgramme retention at 6 and 12 months
Notes

Contact with authors to clarify if withdrawals occurred before or after randomisation

SWEAT2 compared moderate walking programme vs. swimming programme. Primary outcome adherence to the programme

Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Computer generated random numbers by a statistician"
Adequate sequence generation?Yes"Computer generated random numbers"
Blinding?Unclear"Not practical to blind the participants or the research staff to the group assignment" (see other sources of bias below)
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?Yes"Walk and swim sessions were not separated according to the behavioural intervention participants asked not to discuss written materials in the practical sessions"

Dorman 1997

Methods2-arm randomised trial
DataAll UK participants entered into the International Stroke Trial between 2 March 1993 and 31 May 1995 who were still alive
Comparisons

1. Short EuroQol posted with personalised letter and reply paid envelope. 1 reminder sent after 2 weeks

2. Long SF36 questionnaire posted with personalised letter and reply paid envelope. 1 reminder sent after 2 weeks

OutcomesFrequency of response after first and reminder mail out. Data for response to first mail out used
NotesInternational Stroke Trial compared heparin 125,000 IU twice daily + aspirin 300 mg vs. heparin 125,000 IU twice daily; heparin 5000 IU twice daily + aspirin 300 mg daily vs. heparin 5000 IU twice daily vs. aspirin 300 mg daily vs. no heparin or aspirin. Primary outcome: death within 14 days or dependency at 6 months
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthors response "generated by computer"
Adequate sequence generation?UnclearAuthor response "generated by computer"
Blinding?UnclearAuthors report "there was no blinding for either study staff or participants"
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement

Edwards 2001

Methods2-arm randomised trial
DataAdults aged ≥ 16 years with head injury in the UK CRASH trial
Comparisons

1. 1-page, 7-question functional dependence questionnaire with covering letter and stamped return envelope. Reminders after 4 and 8 weeks

2. 3-page, 16-question functional dependence questionnaire with covering letter and stamped return envelope. Reminders after 4 and 8 weeks

OutcomesNo of questionnaires returned within 3 months
Notes

Authors provided numbers randomised and responded

Primary outcome for the CRASH: death from any cause within 2 weeks of injury and death or disability at 6 months

Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Central computer"
Adequate sequence generation?Unclear"Central computer"
Blinding?UnclearAuthor response "questionnaires were packaged and sent to patients by personnel who were independent of the study"
Free of selective outcome reporting?YesReported non-response
Other sources of bias?UnclearInsufficient information to make a judgement

Ford 2006

Methods2-arm quasi-randomised trial
DataAfrican American men aged 55 years enrolled in the intervention screening arm of the PLCO trial
Comparisons1. Indepth case management. Case management monthly telephone calls to participants, assisted making medical appointments, helped participants obtain health insurance information, legal aid, transportation services, food programmes, financial support, medication assistance, free medical care, information related to health risks facing African Americans. Provision of PLCO Cancer Screening Trial screening information and the scheduling of annual screening appointments.
2. Regular trial screening procedures. Participants called annually to schedule screening examinations  
OutcomesNumber completing the next scheduled PLCO cancer screen at 3 years
NotesPLCO trial compared digital rectal examination, transvaginal ultrasound and chest x-ray at baseline and 5 years to usual follow-up. PSA and cancer antigen CA125 at baseline, and annually for 5 years. Primary outcome mortality from 4 PLCO cancers
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearNot described
Adequate sequence generation?No"Randomised by participant id number"
Blinding?UnclearInsufficient information to make a judgement about this
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Gates 2009

Methods2-arm quasi-randomised trial
DataAll participants enrolled in the MINT trial due a follow-up questionnaire at 4 or 8 months after whiplash injury
Comparisons

1. GBP5 voucher redeemable at shops www.highstreetvouchers.com plus questionnaire, cover letter included a sentence explaining that the voucher is to thank participants for their time and effort, delivered by post

2. No voucher and a standard covering letter with the questionnaire

OutcomesNumber of questionnaires returned after first contact with participants
Notes

Number of questionnaires returned in the incentive arm checked with authors

Primary outcome for the Managing Injuries of the Neck Trial (MINT) was returned to normal after whiplash injury measured using the Neck Disability Index (NDI)

Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearAuthor response "lack of concealment of allocations before randomisation was not a major concern because it would have been very difficult for the staff in the study office who were sending out the questionnaires to have selectively allocated systematically different patients to the trial arm"
Adequate sequence generation?NoAuthor response "allocation to study arms was according to whether a specific digit of the patients study number was odd or even"
Blinding?UnclearAuthor response "trial office staff were unblinded, they had no influence over any participant's decision to return the questionnaire, and postal and telephone follow-up contacts were performed in a standardised way for all participants, without any reference to whether or not they were participating in the incentive study"
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Hughes 1989

Methods2-arm randomised trial
DataAll smokers for > 1 year in the nicotine gum vs. placebo gum trial
Comparisons

1. A letter promising a free reprint of the study results in return for sending in the questionnaire

2. No offer of results

OutcomesQuestionnaire response. No time point given
Notes

Author contacted regarding numbers randomised

Nicotine gum trial was double blind. The primary outcome was alleviation of signs and symptoms of tobacco withdrawal measured using a rating list and POMS (profile of mood states) questionnaire and DSM III criteria for tobacco withdrawal

Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "by a non involved researcher sequence in sealed envelope, never opened during study"
Adequate sequence generation?YesAuthor response "using a table of random numbers"
Blinding?UnclearAuthor response "participants fully aware of each condition and which they were in"
Free of selective outcome reporting?YesResponse rates reported
Other sources of bias?UnclearInsufficient evidence to make a judgement on this

Kenton 2007a

MethodsRandomised 2 x 2 factorial trial (incentive vs. offer of incentive)
DataNew mothers with Edinburgh postnatal depression scale score > 9 participating in postpartum depression peer support trial
Comparisons

1. USD2 coin mailed with questionnaire

2. Offer of entry into prize draw for USD50 gift certificate

OutcomesQuestionnaire response. No time point given
NotesThe postpartum depression peer support trial enrolled women < 2 weeks' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID)
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"
Adequate sequence generation?UnclearAuthor response "computer generated"
Blinding?YesAuthor response "participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Kenton 2007a-d

MethodsRandomised 2 x 2 factorial trial
DataNew mothers with Edinburgh, postnatal depression scale score > 9 participating in postpartum depression peer support trial
Comparisons

1. USD2 coin mailed with questionnaire

2. Offer of entry into prize draw for USD50 gift certificate

3. USD2 coin with questionnaire sent with high-priority postage stamp

4. Offer of entry into lottery draw for USD50 gift certificate with high-priority postage stamp

OutcomesQuestionnaire response. No time point given
NotesThe postpartum depression peer support trial enrolled women < 2 weeks' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID)
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"
Adequate sequence generation?UnclearAuthor response "computer generated"
Blinding?YesAuthor response "participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Kenton 2007b

MethodsRandomised 2 x 2 factorial trial (incentive + priority mail vs. offer of incentive + priority mail)
DataNew mothers with Edinburgh postnatal depression scale score > 9 participating in postpartum depression peer support trial
Comparisons

1. USD2 coin with questionnaire sent with high-priority postage stamp

2. Offer of entry into lottery draw for USD50 gift certificate with high-priority postage stamp

OutcomesQuestionnaire response. No time point given
NotesThe postpartum depression peer support trial enrolled women < 2 weeks' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID).
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"
Adequate sequence generation?UnclearAuthor response "computer generated"
Blinding?Yes

Author response "

participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"

Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Kenton 2007c

MethodsRandomised 2 x 2 factorial trial (priority mail vs. no priority mail)
DataNew mothers with Edinburgh postnatal depression scale score > 9 participating in postpartum depression peer support trial
Comparisons

1. USD2 coin sent by high-priority postage stamp plus questionnaire

2. USD2 coin plus questionnaire

OutcomesQuestionnaire response. No time point given
NotesThe postpartum depression peer support trial enrolled women < 2 week' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID)
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"
Adequate sequence generation?UnclearAuthor response "computer generated"
Blinding?YesAuthor response "participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Kenton 2007d

MethodsRandomised 2 x 2 factorial trial (offer of entry into prize draw plus high-priority postage stamp vs. offer of entry into prize draw)
DataNew mothers with Edinburgh postnatal depression scale score > 9 participating in postpartum depression peer support trial
Comparisons

1. Offer of entry into prize draw for USD50 gift certificate plus high-priority postage stamp

2. Offer of entry into prize draw for USD50 gift certificate

OutcomesQuestionnaire response. No time point given
NotesThe postpartum depression peer support trial enrolled women < 2 weeks' postpartum at high risk for postnatal depression. New proactive individualised telephone-based peer support with standard postpartum care. The primary outcome was postnatal depression measured using the Edinburgh postnatal depression scale and Structured Clinical Interview Depression (SCID)
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "allocation sequence stored in a password protected file only accessed by an external researcher, computer generated"
Adequate sequence generation?UnclearAuthor response "computer generated"
Blinding?YesAuthor response "participants were not aware of the sub-study and didn't know other participants were receiving different types of mailings. The trial coordinator was not blinded to the study group"
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Kenyon 2005

Methods2-arm randomised trial
DataWomen in ORACLE 1 and  2 participating in the evaluation of health and development study
Comparisons

1. GBP5 voucher mailed with questionnaire redeemable at many high street shops

2. No incentive

OutcomesQuestionnaire response rate. No time point given
NotesORACLE trial: women with preterm prelabour rupture of fetal membranes and women with intact membranes in preterm labour, randomised to erythromycin 250 mg, co-amoxiclav 325 mg, erythromycin 250 mg plus co-amoxiclav 325 mg or placebo x 10 days or until birth. Primary outcome composite of neonatal death, chronic lung disease or major cerebral abnormality before discharge from hospital
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Randomly assigned by concealed computer generated allocation"
Adequate sequence generation?Yes"Randomly assigned by "computer" "
Blinding?YesAuthor response "both the participants and the study personnel were blinded to the allocation"
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Khadjesari 2011

Methods

Khadjesari 1: 4-arm randomised trial

Khadjesari 2: 2-arm randomised trial

DataSee 'Table of characteristics' for Khadjesari 2011 1abc; Khadjesari 2011 2
ComparisonsSee 'Table of characteristics' for Khadjesari 2011 1abc; Khadjesari 2011 2
OutcomesSee 'Table of characteristics' for Khadjesari 2011 1abc; Khadjesari 2011 2
NotesSee 'Table of characteristics' for Khadjesari 2011 1abc; Khadjesari 2011 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed", "randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesAll outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Khadjesari 2011 1a

Methods4-arm randomised trial (offer of incentives vs. no offer)
DataNon-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test
Comparisons

1. Offer of GBP5 Amazon.co.uk voucher. Emailed voucher code on receipt of response (arm a)

2. Email with no offer of incentive (control)

OutcomesProportion completing questionnaire after 40 days
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Khadjesari 2011 1abc

Methods4-arm randomised trial
DataNon-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test
Comparisons

1. Offer of GBP5 Amazon.co.uk voucher. Emailed voucher code on receipt of response (arm a)

2. Offer of GBP5 donation to Cancer Research UK. Emailed hyperlink to charity's website showing donation when response received (arm b)

3. Offer of entry into GBP250 prize draw emailed confirmation of entry when response received (arm c)

4. Email prompt for completion of questionnaires with no offer of incentive (control arm)

OutcomesProportion completing questionnaire after 40 days
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Khadjesari 2011 1ac

Methods4-arm randomised trial (incentives combined vs. no incentive)
DataNon-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test
Comparisons

1. Offer of GBP5 Amazon.co.uk voucher. Emailed voucher code on receipt of response (arm a)

2.Offer of entry into GBP250 prize draw emailed confirmation of entry when response received (arm c)

3. Email with no offer of incentive (control arm)

OutcomesProportion completing questionnaire after 40 days
Notes

Primary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication

Separate comparison for arm b of attrition trial see Khadjesari 2011 1b

Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Khadjesari 2011 1b

Methods4-arm randomised trial (offer of donation to charity vs. no offer)
DataNon-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test
Comparisons

1. Offer of GBP5 donation to Cancer Research UK. Emailed hyperlink to charity's website showing donation when response received (arm b)

2. Email with no offer of incentive (control)

OutcomesProportion completing questionnaire after 40 days
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Khadjesari 2011 1c

Methods4-arm randomised trial (offer of entry into prize draw vs. no offer)
DataNon-responders at 1 week to the pilot DYD trial 3-month online follow-up questionnaire. Adults aged ≥ 18 scoring 5 or more on the AUDIT-C test
Comparisons

1. Offer of entry into GBP250 prize draw emailed confirmation of entry when response received (arm c)

2. Email with no offer of incentive (control arm)

OutcomesProportion completing questionnaire after 40 days
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Khadjesari 2011 2

Methods2-arm randomised trial
DataAll DYD trial study participants
Comparisons

1. Offer of a GBP10 Amazon.co.uk voucher. Email with voucher code sent on completion of questionnaire

2. Reminder email with no voucher offer

OutcomesThe proportion of participants that completed the questionnaire after 40 days
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed", "randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesAll outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Land 2007

Methods2-arm randomised trial
DataResearch sites participating in the NSABP B35 trial
Comparisons

1. Automated prospective monthly reminder to trial sites of upcoming participant-reported outcome assessments. Reminder listed participants expected to complete Behavioral and Health Outcomes forms in upcoming 3 months

2. No monthly assessment reminder

OutcomesReceipt of questionnaire at any time
NotesB35 Anastrozole vs. tamoxifen for the treatment of ductal carcinoma in situ women aged 55 years plus. Primary outcome: time to first breast cancer reoccurrence
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearAuthor response to question "describe the methods if any, used to conceal the allocation sequence for the prospective reminder study in B35", "not applicable. Institutions were all randomly assigned before trial initiation, so there was no sequence"
Adequate sequence generation?YesAuthor response "prospective reminder in B-35, three strata were generated, and half in each were randomly assigned to receive the reminder"
Blinding?NoAuthor response "there was no blinding the prospective reminder was received by clinical staff"
Free of selective outcome reporting?YesPrimary outcome reported
Other sources of bias?UnclearInsufficient information to make a judgement

Leigh Brown 1997

Methods2-arm randomised trial
DataParticipants in OMENS due to receive follow-up questionnaires between March and December 1995
Comparisons

1. Offer of GBP25 gift voucher monthly prize draw. Postcard reminder after 10 days and 3 weeks with reference to offer

2. No offer. Post card reminders after 10 days and 3 weeks with no reference to offer

OutcomesQuestionnaire response. No time point given
NotesOMENS effectiveness of musculoskeletal medicine vs. care by orthopaedic surgeon-led services for the treatment of non-surgical orthopaedic outpatients Primary outcomes: change in participant reported health (SF-36 and EuroQol) and marginal health
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?NoAuthor response "table of random numbers"
Adequate sequence generation?YesAuthor response "table of random numbers"
Blinding?UnclearAuthor response "those randomised to take part in the prize draw were aware of the intervention and those randomised to be excluded from the draw were unaware. The trial team were aware of the allocation so that they could arrange the monthly draw"
Free of selective outcome reporting?YesDefined outcomes reported outcome reported
Other sources of bias?UnclearInsufficient evidence to make a judgement about this

Letley 2000

Methods2-arm randomised trial
DataParticipants in the UKBEAM feasibility trial, aged 18-65 years with low back pain and a score of 4 or more on the RDQ
Comparisons

Review author (VB) checked which was control and which was intervention

1. RDQ before SF-36in a 26-page questionnaire

2. SF-36 before RDQ in a 26-page questionnaire

OutcomesQuestionnaire response at 3 months
NotesStudy complete, no data available at 8 September 2011. UKBEAM: compared the effectiveness of exercise, manipulation, exercise and manipulation. Primary outcome for UKBEAM scores on the RDQ at 3 and 12 months
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor reported "sequence generation was via remote service ensuring allocation concealment"
Adequate sequence generation?YesAuthor reported "sequence generation was random using randomised permuted blocks"
Blinding?UnclearNo data
Free of selective outcome reporting?YesOverall response rates reported
Other sources of bias?UnclearNo data

MacLennan

Methods2-arm randomised trial
DataParticipants who had not returned questionnaires at 12, 24 and 36 months in the RECORD trial
Comparisons

1. Telephone call from RECORD office before first reminder questionnaire sent. Participant asked to complete questionnaire and to try to fill in all questions. Further telephone call from study nurse or RECORD office if not returned after 3 weeks

2. Repeat mailing of usual follow-up letter and questionnaire. Telephone call by study nurse or RECORD office if not returned after 3 weeks (routine follow-up)

Outcomes

Proportion of first reminder questionnaires returned

Proportion of questionnaires returned at 4 months

Completeness of data

NotesPrimary outcome all new low-energy fractures (self reported) EQ-5D Short form-12. Trial identified through mail out to UK clinical trial units
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "central randomisation service"
Adequate sequence generation?YesAuthor response "randomised to receive the "intervention" or not using the central randomisation service"
Blinding?Yes

Author response "blinding was not possible in the "intervention" arm. The control were blinded. Trial staff phoning were not blinded.

Outcome assessment in the sub-study was objective: questionnaire returned yes/no. Trial staff and trial participants were blinded to the RECORD trial allocation"

Free of selective outcome reporting?YesAll defined outcomes reported

Man 2011

Methods2-arm randomised trial
DataAdults 18-65 years sent a 6-month follow-up questionnaire in a trial of yoga for chronic back pain
Comparisons

1. SMS text message: "Yoga trial: You should receive a questionnaire any day now. The data is important to us so please return it as soon as you can. Many thanks"

2. Email message: "Thank you for taking part in the Yoga for Low Back Pain Trial. This is an automatic reminder. You should receive your six month questionnaire any day now. The answers you give in the questionnaire are very important to the trial. Therefore, we should be most grateful if you would complete and return the questionnaire (and any other documents) as soon as possible please. Thank you"

3. SMS text message plus email message

4. No SMS test message or email message

OutcomesPostal questionnaire response
NotesYoga for chronic back pain trial: primary outcome functional limitations and disability measured by the RDQ. Setting 13 non-national health service settings in the UK. Participants were recruited through general practices
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesComputer-generated sequence to randomly allocate participants
Adequate sequence generation?YesIndependent data manager generated a computer sequence
Blinding?UnclearNo data
Free of selective outcome reporting?YesReported response to questionnaires
Other sources of bias?UnclearNo data

Marson 2007

Methods2-arm randomised trial
DataParticipants in SANAD due a QoL questionnaire at 1 year
Comparisons

1. Cover letter with estimate of the length of time required to complete questionnaire

2. Standard cover letter without estimate of the length of time required to complete questionnaire

OutcomesResponse rate. No time point given
NotesSANAD compared the long-term effects of antiepileptic drugs in people with epilepsy. Participants randomised to arm a received carbamazepine, gabapentin, lamotrigine, oxcarbazepine, or topiramate and valproate, lamotrigine, or arm B received topiramate. Primary outcome time to treatment failure and time to achieve a 12-month remission of seizures
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?NoAuthor response "the list was given to the study researcher who worked sequentially through the list from left to right from top to bottom"
Adequate sequence generation?YesAuthor response "list of allocations was generated electronically using random permuted blocks of length 20"
Blinding?YesAuthor response "this was a postal study, blinding was not necessary for participants; they received either a time-framed letter or non time-framed letter. Clinical personnel did not see the letters, as these were sent postally directly to participants from the study research office"
Free of selective outcome reporting?YesAttrition reported
Other sources of bias?UnclearInsufficient evidence to make a judgement about this

McCambridge 2011

MethodsMcCambridge 1: 4-arm randomised trial in the pilot phase of the DYD trial; McCambridge 2: 4-arm randomised trial in the host DYD trial
DataSee 'Table of characteristics' for McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b
ComparisonsSee 'Table of characteristics' for McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b
OutcomesSee 'Table of characteristics' for McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b
NotesSee 'Table of characteristics' for McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesSee 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b
Adequate sequence generation?YesSee 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b
Blinding?UnclearSee 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b
Free of selective outcome reporting?YesSee 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b
Other sources of bias?UnclearSee 'Risk of bias' table McCambridge 2011 1; McCambridge 2011 1a; McCambridge 2011 1b; McCambridge 2011 2; McCambridge 2011 2a; McCambridge 2011 2b

McCambridge 2011 1

Methods4-arm randomised trial in the pilot phase of the DYD trial
DataAdults aged ≥ 18 years scoring ≥ 5 on the AUDIT-C test participating in the pilot phase of the DYD trial 1-month data used
Comparisons

1. APQ 23 items

2. AUDIT 10

3. LDQ 10

4. CORE-OM 23/34 items (mental health assessment)

OutcomesResponse to electronic questionnaires
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

McCambridge 2011 1a

Methods4-arm randomised trial ('relevance' of questionnaire: alcohol vs. mental health data)
DataAdults aged ≥ 18 years scoring 5 or more on the AUDIT-C test participating in the pilot phase of the DYD trial 1-month data used
Comparisons

1. APQ 23 items

2. CORE-OM 23/34 items (Mental health assessment)

OutcomesResponse to electronic questionnaires
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

McCambridge 2011 1b

Methods4-arm randomised trial (short vs. long questionnaire comparison)
DataAdults aged ≥ 18 years scoring 5 or more on the AUDIT-C test participating in the DYD trial. For this comparison 1-month follow-up data from McCambridge 2011 1 used
Comparisons

1. AUDIT 10 items + LDQ 10 items

2. APQ 23 items

OutcomesResponse to electronic questionnaires
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

McCambridge 2011 2

Methods4-arm randomised trial in the host DYD trial
DataAdults aged ≥ 18 years scoring 5 or more on the AUDIT-C test participating in the pilot phase of the DYD trial 3-month data used
Comparisons

1. APQ 23 items

2. AUDIT 10

3. LDQ 10

4. CORE-OM 23/34 items (Mental health assessment)

OutcomesResponse to electronic questionnaires
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

McCambridge 2011 2a

Methods4-arm randomised trial ('relevance' of questionnaire alcohol vs. mental health)
DataAdults aged ≥ 18 years scoring 5 or more on the AUDIT-C test participating in the DYD trial. For this comparison 3-month follow-up data from McCambridge 2011 2 used
Comparisons

1. AUDIT 10 items + LDQ 10 items

2. CORE-10

OutcomesResponse to electronic questionnaires
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

McCambridge 2011 2b

Methods4-arm randomised trial (short vs. long questionnaire)
DataAdults aged ≥ 18 years scoring 5 or more on the AUDIT-C test participating in the DYD trial. For this comparison, 3-month follow-up data from McCambridge 2011 2 used
Comparisons

1. AUDIT 10 items + LDQ 10 items

2. APQ 23 items

OutcomesResponse to electronic questionnaires
NotesPrimary outcome for DYD: total past week alcohol consumption measured using the TOT-AL measure of total alcohol consumption. Trial identified through personal communication
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Allocation was concealed randomisation could not be subverted by the study team"
Adequate sequence generation?YesComputer-generated randomisation sequence
Blinding?UnclearUnclear
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

McColl 2003

Methods2-arm randomised trial
Data40 adults aged ≥ 18 years with asthma from 62 general practices participating in the COGENT trial
ComparisonsSee 'Table of characteristics' for McColl 2003 1; McColl 2003 2
OutcomesSee 'Table of characteristics' for McColl 2003 1; McColl 2003 2
NotesSee 'Table of characteristics' for McColl 2003 1; McColl 2003 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?NoSee 'Risk of bias' table McColl 2003 1; McColl 2003 2
Adequate sequence generation?UnclearSee 'Risk of bias' table McColl 2003 1; McColl 2003 2
Blinding?YesSee 'Risk of bias' table McColl 2003 1; McColl 2003 2
Free of selective outcome reporting?YesSee 'Risk of bias' table McColl 2003 1; McColl 2003 2
Other sources of bias?UnclearSee 'Risk of bias' table McColl 2003 1; McColl 2003 2

McColl 2003 1

Methods2-arm randomised trial
Data40 adults aged ≥ 18 years with asthma from 62 general practices participating in the COGENT trial
Comparisons

1. Asthma specific instruments (NASQ and AQLQ) followed by generic questions in medical outcomes SF-36 version 1 and EQ-5D

2. Generic questions followed by condition specific

OutcomesQuestionnaire response rates, speed and patterns. No time point given
NotesCOGENT before and after cluster randomised, 2 x 2 factorial trial. Evaluated use of computerised support system for decision making for implementing evidenced-based clinical guidelines for the management of asthma (and angina see McColl 2003 2). General practices randomised to computerised guidelines for the management of angina or asthma. Primary outcome adherence to guidelines
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?NoAuthor response "practices provided us with a computerised list of all patients fulfilling eligibility criteria. We drew a simple random sample of patients from that list first 40 from list received version 1, second 40 version 2"
Adequate sequence generation?UnclearAuthor response "practices provided us with a computerised list of all patients fulfilling eligibility criteria. We drew a simple random sample of patients from that list. First 40 received version 1, second 40 version 2. Logistically, truly random allocation of the 80 patients to versions 1 and 2 would have been impractical to implement"
Blinding?YesAuthor response "respondents were not alerted to the fact that we were experimenting with the order of instruments"
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient evidence to make a judgement about this

McColl 2003 2

Methods2-arm randomised trial
Data40 adults aged ≥ 18 years with asthma from 62 general practices participating in the COGENT trial
Comparisons

1. Angina-specific instruments (SAQ) followed by generic questions (SF-36 version 1 and EQ-5D) 

2. Generic questions followed by condition-specific questions

OutcomesQuestionnaire response rates, speed and patterns. No time point given
NotesCOGENT before and after cluster randomised, 2 x 2 factorial trial. Evaluated use of computerised support system for decision making for implementing evidenced-based clinical guidelines for the management of angina (and asthma see McColl 2003 1). General practices randomised to computerised guidelines for the management of angina or asthma. Primary outcome adherence to guidelines
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?NoAuthor response "practices provided us with a computerised list of all patients fulfilling eligibility criteria. We drew a simple random sample of patients from that list. First 40 from list received version 1, second 40 version 2"
Adequate sequence generation?UnclearAuthor response "practices provided us with a computerised list of all patients fulfilling eligibility criteria. We drew a simple random sample of patients from that list. First 40 received version 1, second 40 version 2. Logistically, truly random allocation of the 80 patients to versions 1 and 2 would have been impractical to implement"
Blinding?YesAuthor response "respondents were not alerted to the fact that we were experimenting with the order of instruments"
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient evidence to make a judgement about this

Nakash 2007

Methods2-arm randomised trial
DataParticipants aged ≥ 16 years from 5 trial centres randomised to CAST between November 2003 and July 2005
Comparisons

1. Trial calendar: monthly customised calendar, included prenotification caption on the months the participant is due to receive the questionnaire with reminder caption the following month

2. No trial calendar

Outcomes

Questionnaire response at 4, 12 weeks and 9 months. Response at 4 weeks used for analysis

Amount of prompting required to return questionnaires at each time point

Percentage of missing data of the core outcome

NotesCAST tubular bandage, below knee cast, Aircast® ankle brace and Bledsoe® boot compared in people with acute severe ankle sprain. Primary outcome FAOS, FLP, SF-12 and EuroQol 5
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Yes"Baseline packs compiled in advance and stored at trial sites. On randomisation the next consecutively numbered pack was taken. Allocation concealed until participant recruited into CAST and pack opened"
Adequate sequence generation?Yes"Computer generated random sequence"
Blinding?Yes"No blinding of participants or clinic staff", "data inputting was blind to allocation"
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient evidence to make a judgement about this

Renfroe 2002a

Methods2 x 2 x 2 x 2 factorial randomised trial (certificate of appreciation vs. no certificate)
DataAll surviving participants at AVID study termination
Comparisons

1. Certificate of appreciation with cover letter signed by physician or co-ordinator sent either by express or standard post, 2-3 weeks after last AVID follow-up visit or 1-4 months after last AVID follow-up visit

2. No certificate with cover letter signed by physician or co-ordinator, sent either by express or standard post, 2-3 weeks after last AVID follow-up visit or 1-4 months after last AVID follow-up visit

OutcomesQuestionnaire response. No time point given
NotesAVID conducted in people resuscitated from ventricular fibrillation or cardioverted for ventricular tachycardia. Compared implanted cardioverter defibrillator vs. antiarrhythmic drugs. Primary outcome overall mortality
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearThe AVID CTC prepared the resulting 16 sets of distinct participant survey packets. These participant-specific packets were then mailed to the study co-ordinators, who distributed them to participants as instructed
Adequate sequence generation?YesAuthor response "all patients in the study were randomised. Details of the randomisation scheme are no longer extant, but given the factorial design I think it's safe to assume that the randomization for each factor was by permuted blocks of size 2"
Blinding?YesParticipants were instructed to mail the completed surveys directly to the CTC in postage-paid, self-addressed return envelopes. Only their AVID study number identified participants, assuring confidentiality of their survey responses
Free of selective outcome reporting?Yes All defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Renfroe 2002a-d

Methods2 x 2 x 2 x 2 factorial randomised trial
DataAll surviving participants at AVID study termination
Comparisons

1. Certificate of appreciation

2. No certificate of appreciation

3. Overnight express delivery

4. Regular post

5. Cover letter signed by physician

6. Cover letter signed by study co-ordinator

7. Early administration

8. Late administration

OutcomesQuestionnaire response. No time point given
NotesAVID conducted in people resuscitated from ventricular fibrillation or cardioverted for ventricular tachycardia. Compared implanted cardioverter defibrillator vs. antiarrhythmic drugs. Primary outcome overall mortality
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearThe AVID CTC prepared the resulting 16 sets of distinct participant survey packets. These participant-specific packets were then mailed to the study co-ordinators, who distributed them to participants as instructed
Adequate sequence generation?YesAuthor response "all patients in the study were randomised. Details of the randomisation scheme are no longer extant, but given the factorial design I think it's safe to assume that the randomization for each factor was by permuted blocks of size 2"
Blinding?YesParticipants were instructed to mail the completed surveys directly to the CTC in postage-paid, self addressed return envelopes. Only their AVID study number identified participants, assuring confidentiality of their survey responses
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Renfroe 2002b

Methods2 x 2 x 2 x 2 factorial randomised trial (express vs. regular post)
DataAll surviving participants at AVID study termination
Comparisons

1. Overnight express delivery with cover letter signed by physician or co-ordinator with or without a certificate of appreciation 2-3 weeks after last AVID follow-up visit or 1-4 months after last AVID follow-up visit

2. Regular post with cover letter signed by physician or co-ordinator with or without a certificate of appreciation 2-3 weeks after last AVID follow-up visit or 1-4 months after last AVID follow-up visit

OutcomesQuestionnaire response. No time point given
NotesAVID conducted in people resuscitated from ventricular fibrillation or cardioverted for ventricular tachycardia. Compared implanted cardioverter defibrillator with antiarrhythmic drugs. Primary outcome overall mortality
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearThe AVID CTC prepared the resulting 16 sets of distinct participant survey packets. These participant-specific packets were then mailed to the study co-ordinators, who distributed them to participants as instructed
Adequate sequence generation?YesAuthor response "all patients in the study were randomised. Details of the randomisation scheme are no longer extant, but given the factorial design I think it's safe to assume that the randomisation for each factor was by permuted blocks of size 2"
Blinding?YesParticipants were instructed to mail the completed surveys directly to the CTC in postage-paid, self-addressed return envelopes. Only their AVID study number identified participants, assuring confidentiality of their survey responses
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Renfroe 2002c

Methods2 x 2 x 2 x 2 factorial randomised trial (physician vs. study co-ordinator signed cover letter)
DataAll surviving participants at AVID study termination
Comparisons

1. Cover letter signed by physician sent either by express or standard post, with or without a certificate of appreciation, 2-3 weeks after last AVID follow-up visit or 1-4 months after last AVID follow-up visit

2. Cover letter signed by study co-ordinator sent either by express or standard post, with or without a certificate of appreciation 2-3 weeks after last AVID follow-up visit or 1-4 months after last AVID follow-up visit

OutcomesQuestionnaire response. No time point given
NotesAVID conducted in participants resuscitated from ventricular fibrillation or cardioverted for ventricular tachycardia. Compared implanted cardioverter defibrillator vs. antiarrhythmic drugs. Primary outcome overall mortality
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearThe AVID CTC prepared the resulting 16 sets of distinct participant survey packets. These participant-specific packets were then mailed to the study co-ordinators, who distributed them to participants as instructed
Adequate sequence generation?YesAuthor response "all patients in the study were randomised. Details of the randomisation scheme are no longer extant, but given the factorial design I think it's safe to assume that the randomisation for each factor was by permuted blocks of size 2"
Blinding?YesParticipants were instructed to mail the completed surveys directly to the CTC in postage-paid, self-addressed return envelopes. Only their AVID study number identified participants, assuring confidentiality of their survey responses
Free of selective outcome reporting?YesReport all outcomes
Other sources of bias?UnclearInsufficient information to make a judgement on this

Renfroe 2002d

Methods2 x 2 x 2 x 2 factorial randomised trial (early vs. late administration of questionnaire)
DataAll surviving participants at AVID study termination
Comparisons

1. Questionnaire sent 2-3 weeks after last AVID follow-up visit by express or standard post with cover letter signed by physician or co-ordinator with or without a certificate of appreciation

2. Questionnaire sent 1-4 months after last AVID follow-up visit, by express or standard post with cover letter signed by physician or co-ordinator with or without a certificate of appreciation

OutcomesQuestionnaire response. No time point given
NotesAVID conducted in people resuscitated from ventricular fibrillation or cardioverted for ventricular tachycardia. Compared implanted cardioverter defibrillator vs. antiarrhythmic drugs. Primary outcome overall mortality
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearThe AVID CTC prepared the resulting 16 sets of distinct participant survey packets. These participant-specific packets were then mailed to the study co-ordinators, who distributed them to participants as instructed
Adequate sequence generation?YesAuthor response "all patients in the study were randomised. Details of the randomisation scheme are no longer extant, but given the factorial design I think it's safe to assume that the randomisation for each factor was by permuted blocks of size 2"
Blinding?YesParticipants were instructed to mail the completed surveys directly to the CTC in postage-paid, self-addressed return envelopes. Only their AVID study number identified participants, assuring confidentiality of their survey responses
Free of selective outcome reporting?YesAll defined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Severi 2011

MethodsTwo 2-arm randomised trials
DataUK smokers aged ≥ 16 years participating in the Txt2stop trial
ComparisonsSee 'Table of characteristics' for Severi 2011 1 and Severi 2011 2
OutcomesSee 'Table of characteristics' for Severi 2011 1 and Severi 2011 2
NotesSee 'Table of characteristics' for Severi 2011 1 and Severi 2011 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear"Concealed from the investigators"
Adequate sequence generation?Yes"Allocated through minimisation using minim software"
Blinding?Yes"Single blind controlled trial, with those recording and assessing outcomes blind to the intervention"
Free of selective outcome reporting?YesAll defined outcomes reported

Severi 2011 1

Methods2-arm randomised trial
DataUK smokers aged ≥16participating in the Txt2stop trial
Comparisons

1. Text message "Be proud of yourself for helping medical research! Thank you for filling in the txt 2 stop questionnaire" plus a fridge magnet: the message on the fridge magnet was placed in a sealed envelope, this said: "medical research is important to society" and pointed out that by taking part in TxT2stop the participants are benefiting society. Fridge magnet sent by post 16-20 weeks after randomisation. Text message sent 3 days after TxT2stop postal follow-up questionnaire

2. Text message reminding the participant the follow-up questionnaire was due 3 days after the TxT2stop questionnaire had been sent. The text message said "Thank you for filling in the TxT2stop questionnaire". Sent 3 days after the text to stop postal follow-up questionnaire

Outcomes

Primary outcome: completed follow-up questionnaires at 30 weeks from randomisation

Secondary outcome: completed follow-up questionnaires at 26 weeks from randomisation

NotesTxt2stop UK-based smoking cessation trial evaluating the effectiveness of the Txt2stop mobile phone text messaging smoking cessation programme on biochemically verified continuous smoking abstinence at 6 months. Compared Txt2stop motivational messages vs. behaviour change support to text messages unrelated to quitting
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear"Concealed from the investigators"
Adequate sequence generation?Yes"Allocated through minimisation using minim software"
Blinding?Yes"Single blind controlled trial, with those recording and assessing outcomes blind to the intervention"
Free of selective outcome reporting?Yes All defined outcomes reported

Severi 2011 2

Methods2-arm randomised trial
DataUK smokers aged ≥ 16 years participating in the Txt2stop trial
Comparisons

1. Telephone call from female principal investigator (senior clinician and researcher) to participants 6 weeks overdue returning specimen to increase participant follow-up plus standard Txt2stop follow-up procedures

2. Standard Txt2stop follow-up procedures

OutcomesCompleted cotinine sample follow-up at the end of May 2009 for Txt2stop (1 month after a telephone call)
NotesTxt2stop UK-based smoking cessation trial evaluating the effectiveness of the Txt2stop mobile phone text messaging smoking cessation program on biochemically verified continuous smoking abstinence at 6 months. Compared Txt2stop motivational messages vs. behaviour change support to text messages unrelated to quitting
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear"Concealed from the investigators"
Adequate sequence generation?Yes"Allocated through minimisation using minim software"
Blinding?Yes"Single blind controlled trial, with those recording and assessing outcomes blind to the intervention"
Free of selective outcome reporting?YesAll defined outcomes reported

Sharp 2006a

Methods2 x 2 x 2 factorial randomised trial (trial-branded pen vs. no pen: comparison 1)
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 months' follow-up
Comparisons

1. TOMBOLA-branded pen + despatch first-class post + second-class stamped preaddressed return envelope

2. No TOMBOLA-branded pen + despatch first-class post + second-class stamped preaddressed return envelope

OutcomesResponse rates at any time
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women aged 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Sharp 2006a-h

Methods2 x 2 x 2 factorial randomised trial
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 months' follow-up
Comparisons

1. TOMBOLA-branded pen + despatch first-class post + second-class stamped preaddressed return envelope

2. No TOMBOLA-branded pen + despatch first-class post + second-class stamped preaddressed return envelope

3. TOMBOLA-branded pen + despatch first-class post + free post business-reply envelope

4. No TOMBOLA-branded pen + despatch first-class post + free post business-reply envelope

5. TOMBOLA-branded pen + despatch second-class post + second-class stamped preaddressed return envelope

6. No TOMBOLA-branded pen + despatch second-class post + second-class stamped preaddressed return envelope

7. TOMBOLA-branded pen + despatch second-class post + free post business-reply envelope

8. No TOMBOLA-branded pen + despatch second-class post + free post business-reply envelope

OutcomesResponse rates at any time
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women aged 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Sharp 2006b

Methods2 x 2 x 2 factorial randomised trial (trial-branded pen vs. no pen: comparison 2)
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 months' follow-up
Comparisons

1. TOMBOLA-branded pen + despatch first-class post + free post business-reply envelope

2. No TOMBOLA-branded pen + despatch first-class post + free post business-reply envelope

OutcomesResponse rates at any time
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women aged 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Sharp 2006c

Methods2 x 2 x 2 factorial randomised trial (trial branded pen vs. no pen: comparison 3)
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 months' follow-up
Comparisons

1. TOMBOLA-branded pen + despatch second-class post + second-class stamped preaddressed return envelope

2. No TOMBOLA-branded pen + despatch second-class post + second-class stamped preaddressed return envelope

OutcomesDefined outcomes reported
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women aged 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Sharp 2006d

Methods2 x 2 x 2 factorial randomised trial (trial-branded pen vs. no pen: comparison 4)
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 month follow-up.
Comparisons

1. TOMBOLA-branded pen + despatch second-class post + free post business-reply envelope

2. No TOMBOLA-branded pen + despatch second-class post + free post business-reply envelope

OutcomesResponse rates at any time
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women aged 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Sharp 2006e

Methods2 x 2 x 2 factorial randomised trial (first vs. second-class post comparison 1)
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 months' follow-up.
Comparisons

1. TOMBOLA-branded pen + despatch first-class post + second-class stamped preaddressed return envelope

2. TOMBOLA-branded pen + despatch second-class post + second-class stamped preaddressed return envelope

OutcomesDefined outcomes reported
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Sharp 2006f

Methods2 x 2 x 2 factorial randomised trial (first vs. second-class post comparison 2)
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 months' follow-up
Comparisons

1. No TOMBOLA-branded pen + despatch first-class post + second-class stamped preaddressed return envelope

2. No TOMBOLA-branded pen + despatch second-class post + second-class stamped preaddressed return envelope

OutcomesResponse rates at any time
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Sharp 2006g

Methods2 x 2 x 2 factorial randomised trial (first vs. second-class comparison 3)
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 months' follow-up
Comparisons

1. TOMBOLA-branded pen + despatch first-class post + second-class stamped preaddressed return envelope

2. TOMBOLA-branded pen + despatch first-class post + free post business-reply envelope

OutcomesResponse rates at any time
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women aged 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesDefined outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Sharp 2006h

Methods2 x 2 x 2 factorial randomised trial (first vs. second-class comparison 4)
DataWomen due to receive a TOMBOLA psychosocial questionnaire during June-August 2003 for 12, 18, 24, 30, 34, 36 month follow-up.
Comparisons

1. No TOMBOLA-branded pen + despatch first-class post + second-class stamped preaddressed return envelope

2. No TOMBOLA-branded pen + despatch first-class post + free post business-reply envelope

OutcomesResponse rates at any time
NotesTOMBOLA compared 2 management policies (colposcopy vs. 6-monthly smears) in women 20-59 years with low-grade abnormal cervical smear. Primary outcome: cumulative incidence of cervical intraepithelial neoplasia grade ≥ 2
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?Unclear2 authors did randomisation
Adequate sequence generation?Unclear"Computer randomised"
Blinding?UnclearOutcome not influenced by lack of blinding
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement on this

Subar 2001

Methods2-arm randomised trial
DataParticipants aged 55-74 years from 3 centres participating in PLCO trial (control arm)
Comparisons

1. Diet history questionnaire DHQ (36-page food frequency questionnaire) plus 1-page questionnaire on time taken to complete questionnaire

2. PLCO food frequency questionnaire (16-page food frequency questionnaire) plus 1-page questionnaire on time taken to complete questionnaire

OutcomesResponse rate. No time point given
NotesPLCO trial compared digital rectal examination, transvaginal ultrasound and chest x-ray at baseline and at 5 years vs. usual follow-up. PSA and cancer antigen CA125 at baseline, and annually for 5 years. Primary outcome: mortality from PLCO cancers
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?NoAuthor response "none known"
Adequate sequence generation?UnclearAuthor response "at each centre half were randomised to received". Authors contacted "we would have likely used some computer generated randomisation scheme"
Blinding?UnclearAuthor response "the respondents were not blinded. Not known if personnel were 'blinded'"
Free of selective outcome reporting?YesExpected outcome reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Sutherland 1996

Methods2-arm randomised trial
Data226 women taking part in the feasibility study for the Canadian diet and breast cancer prevention feasibility trial
Comparisons

1. Total design method for postal follow-up: white envelope with hospital logo and commemorative stamp; headed notepaper; reply self addressed stamped envelope enclosing contents, hand signature on letters. Postcard sent after 7 days, reminders sent twice

2. Customary method for postal follow-up: brown envelope with return address stamped on, computer-printed labels, no signature on letter, reply self addressed stamped envelope folded and inserted behind forms, no reminder

OutcomesTime to return of questionnaire at 70 days
NotesCanadian diet and breast cancer prevention feasibility trial compared teaching women aged over 30 years of age with at least 50% of breast volume occupied by radiological changes of dysplasia how to reduce dietary fat to a level of 15% of calories vs. no teaching. Primary outcome recruitment and retention
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?UnclearNot clear
Adequate sequence generation?YesAuthor response "computer generated random numbers"
Blinding?UnclearAuthor response "personel knew the allocation a subject had received but their only contact with subjects was the follow-up phone call in some allocated to the "customary" method"
Free of selective outcome reporting?YesExpected outcome reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Svoboda 2001

Methods2-arm randomised trial
DataAdults ≥ 16 years with head injury in the CRASH trial (Czech)
Comparisons

1. 1-page, 7-question functional dependence questionnaire sent with a covering letter and a stamped return envelope. Reminders sent after 4 and 8 weeks

2. 3-page, 16-question functional dependence questionnaire sent with a covering letter and a stamped return envelope. Reminders sent after 4 and 8 weeks

OutcomesNumber of questionnaires returned within 3 months
NotesNumbers randomised and responded provided by authors. Primary outcome for the CRASH trial death from any cause within 2 weeks of injury and death or disability at 6 months
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "central computer"
Adequate sequence generation?YesAuthor response "random allocation central computer"
Blinding?UnclearAuthor response "the questionnaires were packaged and sent to patients by personnel who were independent of the study"
Free of selective outcome reporting?YesExpected outcome reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Tai 1997

  1. a

    APQ-23: Alcohol Problems Questionnaire; AQLQ: Asthma Quality of Life Questionnaire; AUDIT-C: Alcohol Use Disorders Identification Test; AVID: Antiarrhythmics Versus Implantable Defibrillators; CTC: Clinical Trial Centre; COGENT: COlorectal cancer GENeTics; CORE OM: CORE Outcome Measure; CRASH: Cortiocosteriod Randomisation after Significant Head injury trial; DYD: Down Your Drink; ELISA: enzyme-linked immunosorbent assay; FAOS: Foot and Ankle Outcome Score; FLP: Functional Limitations Profile; LDQ: Leeds Dependency Questionnaire; MIDAS: MIgraine Disability Assessment Questionnaire; NASQ Newcastle Asthma Symptoms Questionnaire; NSABP: National Surgical Adjuvant Breast and Bowel project; PLCO: Prostate, Lung, Colorectal, Ovarian; PSA: prostate-specific antigen; QoL: quality of life; RDQ: Roland Disability Questionnaire; RECORD: Randomised Evaluation of Calcium Or Vitamin D; SANAD: Standard And New Antiepileptic Drugs; SF-36: Short form-36-item health survey; TOMBOLA: Trial Of Management of Borderline and Other Low-grade Abnormal smears; UKBEAM: UK Back pain Exercise And Manipulation.

Methods2-arm randomised trial
DataAll host study participants lost to follow-up in the evaluation of general practice computer templates trial
Comparisons

1. Recorded delivery reminder letter with QoL questionnaire included, sent once

2. Telephone calls repeated up to 3 times at 10-12 a.m. or 2-5.30 p.m. Message left on answering machine after third call

OutcomesNumber of questionnaires returned. No time point given
NotesEvaluation of general practice computer templates cluster randomised trial compared computerised templates for asthma and diabetes management in general practice. Primary outcome frequency of use of computer templates assessed by examining computerised records of those who responded to QoL questionnaires
Risk of bias
ItemAuthors' judgementDescription
Allocation concealment?YesAuthor response "computer generated by statistician independent to trial manager"
Adequate sequence generation?UnclearAuthor response "computer generated by statistician"
Blinding?UnclearStudy personnel were not blind to the intervention the participants received
Free of selective outcome reporting?YesExpected outcomes reported
Other sources of bias?UnclearInsufficient information to make a judgement about this

Characteristics of excluded studies [ordered by study ID]

StudyReason for exclusion
Arnevik 2009This retention trial was not embedded in a randomised trial
Atherton 2010Comparison of Internet vs. postal questionnaires not randomised
Barry 1996Retention trial compared distribution of scores for subjects completing different questionnaire versions. Author confirmed retention/questionnaire return was not an outcome measure
Bednarek 2008Retention trial outcome is continuation of treatment
Cox 2003Retention trial outcome treatment compliance
Day 1998Retention trial measured adherence to treatment. Authors do not have retention data
Eaker 2004Retention trial embedded in a cohort
Edelstein 2005Retention study is not a randomised trial. Incentives not randomised. Author confirmed these were not instituted to help with retention but with adherence to pill taking and life style modification requirements
Grabowski 1995Substudy aim is retention in treatment comparing different follow-up schedules for addiction treatment trial
Hall 1975Not a randomised/quasi-randomised retention trial
Hall 1978Not a randomised/quasi-randomised retention trial
Hoffman 1998Retention trial embedded in a blood bank cohort
Hopkins 1983Retention trial embedded in a survey
Iglesias 2000Retention trial embedded in a cohort of general practitioner practice participants
Iglesias 2001Retention trial embedded in the recruitment phase of the host trial
Johnson 2004Retention study not embedded in a randomised trial
Katz 2001Retention study is not a randomised trial. Authors confirmed the effectiveness of gift incentives was not evaluated in a substudy for the Pride in Parenting trial
Leidy 2000Retention study appears to be a randomised trial but no response from authors to establish if retention was an outcome. For the substudy, trial sites randomised to 1 of 2 orders of administration of quality of life questionnaires. Response rates not reported. Missing data, internal consistency reliability, mean score values, relationship between the 2 measures evaluated
Marsh 1999Host study was not a randomised trial. "Practices were randomly allocated to the intervention group using random number tables. Each intervention practice matched with one control practice"
McAuley 1994Retention study is not a randomised trial. There is a single randomisation stratified by classes in the morning and early evening. No response from authors regarding randomisation to class times
McBee 2009Retention study not a randomised trial. Authors confirm strategies to improve retention were not evaluated in an Age-Related Eye Disease Study 2 (AREDS2) substudy
Poling 2006Substudy aim is about diagnostic compliance. 4-arm trial comparing contingency management with or without active bupropion and voucher control with or without active bupropion. Here contingency management and voucher control are aimed at getting information on the disease condition/response to treatment for the primary outcome of the host trial i.e. negative urine sample for cocaine and opioids. Contingency management and voucher control are not related to retention in the host trial but related to diagnostic compliance
Puffer 2004Retention RCT was embedded in a survey. Authors confirmed that the 2 x 2 factorial study testing four different questionnaire designs was embedded in a survey.
Rhoades 1998Substudy retention in treatment. 2 x 2 trial of dose and visit frequency of attending a clinic either 2 or 5 days per week. Primary outcome was retention in treatment for all randomizations. Similar to Grabowski 1995 trial
Roberts 2000Retention trial embedded in a survey about menopause services
Schmitz 2005Substudy about compliance to treatment and pill taking behaviour rather than trial retention
Smeeth 2001abSubstudy about response to baseline assessment
Stoner 1998Retention study was not a randomised trial. Host study was a cluster randomised trial. Effectiveness of vouchers not evaluated in a substudy
Tassopoulos 2007Not a retention randomised trial
Wu 1997Substudy designed to evaluate whether scores are different using 3 modes of questionnaire administration, rather than retention

Characteristics of ongoing studies [ordered by study ID]

Land 2007 2

Trial name or titleRandomised Evaluation of NSABP BAHO Compliance Initiatives
MethodsRandomised trial embedded in Protocol B36
DataParticipants or trial sites/institutions participating in the B36 trial. Pre- and postmenopausal women aged > 18 years, with histologically confirmed invasive breast adenocarcinoma
Comparisons

Menstrual history calendar

Usual procedures

OutcomesTo improve form submission compliance in Behavioural and Health Outcomes (BAHO) protocols
Starting dateUnclear
Contact informationLand@nsabp.pitt.edu
Notes

Study identified through personal correspondence with author

B36: US-based comparison of 2 combination chemotherapy regimens to treat women with breast cancer

Mitchell

Trial name or titleA Randomised Controlled Trial of Combined Pre-Contact and Participant Update for Increasing Questionnaire Response Rates in Older Women
MethodsRandomised trial embedded in the SCOOP screening trial
DataWomen aged 70-85 years not receiving treatment for osteoporosis
Comparisons

1. Prenotification to complete the 2-year follow-up questionnaire plus a study update

2. No study update

OutcomesResponse to postal questionnaires at 2-year follow-up
Starting date2010
Contact informationnatasha.mitchell@york.ac.uk
Notes

Study identified through mail out to UK clinical trials units

SCOOP: UK-based pragmatic randomised controlled trial of the effectiveness of screening for osteoporosis in older women for the prevention of fractures. 10-year absolute risk of fracture calculated from a World Health Organization algorithm based on screening questionnaire data, x-ray and dual-energy x-ray absorptiometry scan results compared with usual care

Ancillary