SEARCH

SEARCH BY CITATION

Keywords:

  • randomised trial;
  • cluster randomisation;
  • health services research;
  • implementation research;
  • HIV ;
  • research methods

Health services in Africa have a very severe shortage of doctors and nurses with fewer than 10 doctors per 100 000 population in several countries (World Health Organization 2006), and access to health services is difficult for many patients because of high transport costs (Govindasamy et al. 2012). Despite these constraints, and the limited experience in delivering chronic care in Africa, antiretroviral therapy (ART) has been scaled up rapidly and about 8 million people are now on treatment (World Health Organization 2013a). In most countries, the HIV services are delivered as stand-alone vertical programmes (Munderi et al. 2012).

Non-communicable diseases (NCDs) also require chronic care, and their burden is rising rapidly in Africa (World Health Organization 2009; Lim et al. 2012). The demands for delivering chronic care services will increase substantially (Alleyne et al. 2013), but African health systems are geared towards the control of acute infections (Atun et al. 2013). Research on how to organise and deliver chronic care services in Africa will be essential in order to target the scarce resources efficiently (Ebrahim et al. 2013). Such research has to be integrated into health systems because its central aim is usually to estimate the effectiveness of models of health service delivery under near-normal conditions in order that the findings can be generalised immediately. This is in contrast to many efficacy trials (e.g. of drugs and vaccines), which are usually implemented under parallel systems. We have previously examined the use of cluster randomisation for a vaccine efficacy trial (Jaffar et al. 1999) and examined the operational and ethical issues of integrating research into routine health service delivery (Jaffar et al. 2008). Here we focus on the study design challenges, with a particular focus on whether such trials should be cluster- or individually randomised.

We examine the issues using our experiences from two trials. The REMSTART trial was designed to evaluate the effectiveness of a complex health service intervention in reducing mortality among HIV-infected patients who presented with very low CD4 count to 6 government clinics in Dar es Salaam, Tanzania, and Lusaka, Zambia (ISCRTN20410413). Enrolment began in February 2012 and ended in September 2013. Follow-up is scheduled to end in September 2014. Initially, only patients with CD4 count <100/μl were eligible for enrolment, but the criterion was changed during the course of the trial to enrol any patient presenting with <200 CD4 cells/μl because of slow recruitment and because of the increasing recognition that such patients had a high risk of death. Just prior to that, the criteria for initiating antiretroviral therapy changed from initiation at CD4 count<200/μl to initiation at CD4 count<350/μl. The World Health Organization has recently recommended that antiretroviral therapy should now be initiated at CD4 count<500 cells/μl (World Health Organization 2013b). The change in recruitment criteria of the REMSTART trial was implemented in September 2012 in Zambia and in December 2012 in Tanzania.

The REMSTART trial intervention comprised (i) rapid initiation of antiretroviral therapy, (ii) screening for cryptococcal meningitis using a novel antigen test, (iii) weekly home visits for 4 weeks by trained lay-workers and iv) rescreening for tuberculosis using the Xpert®MTB/RIF assay (Cepheid, Sunnyvale, USA) at about 6 weeks after initiation of ART. Patients were randomised to either this intervention strategy or to standard clinic-based HIV care including ART. All participants were offered screening for tuberculosis at baseline using the Xpert®MTB/RIF assay irrespective of whether they had any symptoms or not. Participants were followed up for 12 months after enrolment. The primary endpoint was all-cause mortality.

The Jinja trial compared a home-based with a facility-based HIV care strategy in Jinja, Uganda, a predominantly rural setting (Jaffar et al. 2009). The home-based strategy involved trained lay-workers visiting the patient at home on a monthly basis. ART was provided by The AIDS Support Organisation (TASO), a large non-governmental organisation. The vast majority of HIV-infected patients in the Jinja district accessed ART services at TASO. There was little provision for ART in government facilities at the time (the TASO clinic was based within the grounds of the Jinja District Hospital). Enrolment into the trial ended in December 2006, and follow-up continued until January 2009.

In both trials, lay-workers received a small salary and were supervised by clinicians and nurses based at the clinics. They received classroom training at the beginning (4 weeks in the Jinja trial and 2 weeks in REMSTART trial) and on-the-job training subsequently. In the home, they delivered drugs, provided adherence support and monitored the participants for adverse events using a checklist. They referred patients if indicated and phoned a clinician based at the clinic when they were uncertain about referral. In the Jinja trial, lay-workers travelled on motorbikes, while in REMSTART trial, they travelled mostly by foot and public transport.

How should health service delivery trials be randomised?

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References

Because health care is delivered to groups (e.g. catchment populations of health centres), such trials are normally cluster-randomised with all participants in a defined cluster receiving the same mode of care – either the intervention or the control strategy (Hayes & Moulton 2009). Typically, 6 or more clusters are randomised to each arm (Hayes & Bennett 2009). This design mimics the real-life situation – it is how health care would normally be delivered.

One major challenge in health service trials is that patients or the healthcare personnel might have strong views on how health care should be delivered. For example, clinic-based care can incur substantial transport costs, and home-based care involves disclosure of HIV status and stigma. In Jinja, Uganda, a single clinic visit cost the equivalent of an average of 13% of a man's and 20% of a woman's monthly wage and took a day of the patient's time (Jaffar et al. 2009), and 19 people in the home-based care arm versus only 3 in the facility arm refused to join for fear of increased stigma. We considered that individual randomisation would be a major challenge in Jinja, but the concept of neighbourhoods having the same mode of care might be acceptable. Because of similar concerns over consent, trials of breastfeeding practices have not been randomised, and instead, the breastfeeding groups have been defined by the choices made by the women (Coovadia et al. 2007). Although there might have been no alternative, the evidence from such studies is weaker than that from randomised trials.

One reason for randomising by clusters is to avoid interaction between people receiving different modes of care (known as contamination). For example, adherence messages delivered to people on one arm could be passed to people in the other arm, thus diluting the efficacy of the intervention. In Jinja, the TASO clinic provided the vast majority of antiretroviral therapy within a 100-km radius for several years, and cluster randomisation ensured separation in the community of people receiving different models of care. However, HIV care is now available widely in Africa, and achieving that separation between people on antiretroviral therapy is no longer possible. We decided to randomise the REMSTART trial individually in the belief that contamination was not a major issue because the intervention had only one behavioural component – adherence support delivered by a trained lay-worker – and contamination between participants might not influence the effects of this. Also the chances of neighbours being in the trial in different trials arms were small (we planned to enrol about 2500 participants from a total catchment population of well over 150 000 urban adults), and the chances of contamination at clinic visits would be minimised given the high degree of activities at each clinic.

Statistical power considerations

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References

Complex interventions involve multiple components, making sample size difficult to estimate. A further complication is that trial conditions can change with changes in management guidelines or practices. Thus, it is essential that assumptions underlying the trial are reviewed periodically and trial size is adjusted where this is indicated.

Increasing the trial size in an individually randomised trial is usually relatively straightforward. In cluster-randomised trials, the number of clusters is usually the major determinant of statistical power, and adding new clusters during the course of a trial is often impractical. This is a significant drawback with cluster randomisation. Sample size calculations for cluster-randomised trials also require an estimate of the variability between clusters (e.g. the coefficient of variation), which is rarely known in advance.

Table 1 shows the sample size calculations for the REMSTART trial as designed and if instead this was cluster-randomised assuming coefficient of variation to be 0.1 or 0.2(Hayes & Bennett 2009). Cluster randomisation requires large increases in sample size to achieve the same level of power.

Table 1. Total number of participants needed in both groups combined in a two-arm randomised trial to detect a 40% reduction in mortality at 90% power and 5% significance level
Mortality (percentage per year)If trial is individually randomisedIf trial is cluster-randomised
Control armIntervention arm12 clusters24 clusters
k = 0.2k = 0.1k = 0.2k = 0.1
  1. k is the coefficient of variation.

84.8262411005383342383116
106.021008804306733912493
127.217507337255628252077
148.415006289219124221780

Blinding in health service trials

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References

Blinding is rarely possible in trials of healthcare delivery, whether they are randomised individually or by clusters, and this can be a major source of bias. It is vital that both researchers and healthcare staff assume equipoise; not doing so will affect the implementation of the intervention and the measurement of outcomes.

It is important that healthcare staff are informed about the wider aspects of research, including its need and uses and that they feel that they have ownership of the research programme. It is essential that healthcare staff understand the concepts of bias and the need for equipoise. Providing training in research methods alone is not enough in such settings.

Control for confounding

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References

In large, individually randomised trials, factors that are predictive of the outcome tend to be equally distributed between the trial arms. In cluster-randomised trials, the number of clusters is usually limited and imbalance between two arms is common. For example, in the Jinja trial, 22 clusters were randomised to each strategy – 44 clusters in total (Jaffar et al. 2009). By chance, the number enrolled differed substantially between the two arms (859 in the home-based care arm compared with 594 for facility), and median baseline CD4 count was significantly lower in the home-based arm. The difficulty in health service trials is that some confounders may be unknown and key known confounders such as socio-demographic variables, access to the clinic, income and ability to afford transport are all difficult to measure; therefore, imbalances between arms are difficult to assess and difficult to adjust in analyses.

Data collection

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References

Data collected by healthcare staff might be of poorer quality than those collected by researchers, as their first priority is to attend to the clinical needs of the patient. Despite this, it is critical that there is zero tolerance of incomplete or inaccurate data. In the REMSTART trial, only essential data in simplified form were collected by clinicians, and independent monitoring of the data was usually performed within about 30 minutes of the patient emerging from the consultation and while the patient is still in clinic so that queries could be resolved.

Data collection can be especially problematic in cluster-randomised trials, as extensive data need to be collected to adjust for the possibility of confounding. The risk of confounding is negligible in large-scale individually randomised trials, and so much greater focus can be placed on the measurement of essential outcome data.

Consistency in the delivery of standard care and in the implementation of the intervention strategy

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References

Healthcare delivery trials require a comparison between standard care and the intervention. However, standard care often varies between Ministry of Health guidelines and practices within clinics. Bringing change and implementing a new strategy (i.e. the intervention arm) in busy, overstretched facilities brings further variation. In an individually randomised trial, standardisation is easier to achieve because fewer clinics are involved than in a cluster-randomised trial. In a cluster-randomised trial, resources are required in each clinic to standardise delivery and to monitor to what extent delivery is in accordance with protocol.

The danger in individually randomised trials is that if healthcare workers can see components of an intervention working well, they might be tempted to introduce them for control subjects; similarly, poorly functioning components of an intervention might be dropped. In cluster randomisation, there is much less interaction between healthcare staff in different facilities, such that different models of care can coexist for longer.

In the REMSTART trial, rapid initiation of ART was perceived to be working well in the intervention arm, and at the same time, there was pressure to increase the number on ART to meet targets; consequently, the practices within the clinics changed in both countries to initiate ART rapidly in both arms of the trial. Had the trial been cluster-randomised, this change in practice may have taken longer.

Conclusions

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References

Trials to address health service delivery questions provide valuable information to guide healthcare delivery, but pose major challenges for health services and researchers. Blinding is rarely possible. Cluster-randomised trials mimic more closely the real-life setting of how care is normally delivered, but power is reduced, and control for confounding and a standardised delivery at the various clinics are more difficult to achieve. Interaction between trial participants and non-trial participants receiving different modes of care is unavoidable for chronic care services, which are available in multiple settings. Control for confounding is challenging in cluster-randomised trials. Accurate data on potential confounders have to be collected, but extensive data collection is impractical in busy clinic settings. In any case, the confounders in health services research are not well understood or difficult to measure. Partnerships between researchers, healthcare workers, public health staff and patient groups are essential in health systems research.

Acknowledgements

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References

We thank the study participants and staff involved with the JINJA and REMSTART trials and our funders, EDCTP (European and Developing Countries Clinical Trials Partnership), United States Centers for Disease Control and Prevention and the United Kingdom Medical Research Council.

References

  1. Top of page
  2. How should health service delivery trials be randomised?
  3. Statistical power considerations
  4. Blinding in health service trials
  5. Control for confounding
  6. Data collection
  7. Consistency in the delivery of standard care and in the implementation of the intervention strategy
  8. Conclusions
  9. Acknowledgements
  10. References
  • Alleyne G, Binagwaho A, Haines A et al. (2013) Embedding non-communicable diseases in the post-2015 development agenda. Lancet 381, 566574.
  • Atun R, Jaffar S, Nishtar S et al. (2013) Improving responsiveness of health systems to non-communicable diseases. Lancet 381, 690697.
  • Coovadia HM, Rollins NC, Bland RM et al. (2007) Mother-to-child transmission of HIV-1 infection during exclusive breastfeeding in the first 6 months of life: an intervention cohort study. Lancet 369, 11071116.
  • Ebrahim S, Pearce N, Smeeth L, Casas JP, Jaffar S & Piot P (2013) Tackling non-communicable diseases in low- and middle-income countries: is the evidence from high-income countries all we need? PLoS Medicine 10, e1001377.
  • Govindasamy D, Ford N & Kranzer K (2012) Risk factors, barriers and facilitators for linkage to antiretroviral therapy care: a systematic review. AIDS 26, 20592067.
  • Hayes RJ & Bennett S (2009) Simple sample size calculation for cluster-randomized trials. International Journal of Epidemiology 28, 319326.
  • Hayes RJ & Moulton LH (2009). Cluster Randomised Trials. Chapman & Hall, London.
  • Jaffar S, Leach A, Hall AJ et al. (1999) Preparation for a pneumococcal vaccine trial in The Gambia: individual or community randomisation? Vaccine 18, 633640.
  • Jaffar S, Amuron B, Birungi J et al. (2008) Integrating research into routine service delivery in an antiretroviral treatment programme: lessons learnt from a cluster randomized trial comparing strategies of HIV care in Jinja, Uganda. Tropical Medicine & International Health: TM & IH 13, 795800.
  • Jaffar S, Amuron B, Foster S et al. (2009) Rates of virologic failure in patients treated in a home-based versus a facility-based HIV-care model in Jinja, southeast Uganda: a cluster-randomised equivalence trial. Lancet 374, 20802089.
  • Lim SS, Vos T, Flaxman AD et al. (2012) A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380, 22242260.
  • Munderi P, Grosskurth H, Droti B & Ross DA (2012) What are the essential components of HIV treatment and care services in low and middle-income countries: an overview by settings and levels of the health system. AIDS 26 (Suppl. 2), S97S103.
  • World Health Organization (2006). World Health Report 2006 - working together for health. World Health Organization, Geneva.
  • World Health Organization (2009). Global health risks: mortality and burden of disease attributable to selected major risks. World Health Organization, Geneva.
  • World Health Organization (2013a). Global update on HIV treatment 2013: results, impact and opportunities. World Health Organization, Geneva.
  • World Health Organization (2013b). Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection – recommendations for a public health approach. World Health Organization, Geneva, Switzerland.