SEARCH

SEARCH BY CITATION

Randomised trials are the most reliable method of comparing treatments and, when properly performed, provide highly persuasive evidence on which to base practice1. Unfortunately, most are too small to provide reliable estimates of effects on substantive measures of outcome such as mortality or major disability. This article, the first of three linked editorial commentaries, will discuss why simpler, more powerful trials are needed to assess the impact of treatment on outcomes such as mortality and major morbidity. The second will examine whether patients can participate in more than one trial concurrently, a key concern for all who contribute to randomised studies. The third will argue the need to measure long term outcomes in the fetus or baby.

The contribution of meta-analysis

  1. Top of page
  2. The contribution of meta-analysis
  3. Disadvantages of meta-analysis
  4. Meta-analysis as a stimulus to larger trials
  5. How can we achieve more powerful trials?
  6. Conclusions
  7. References

All trials are open to at least two types of error: 1. wrongly concluding that a treatment is effective or overestimating its true effect, and 2. wrongly concluding that a treatment does not work. Errors of both types are more likely if trials lack adequate size and power. One method for overcoming these errors arises from Archie Cochrane's call for a critical summary, or overview, of the evidence from all known randomised trials in each specialty. A striking example of this method in perinatal medicine was the overview, or meta-analysis, of trials of antenatal corticosteroids published in this Journal by Crowley et al. in 19902. It synthesised the results of 12 trials in over 3000 women at risk of preterm delivery and showed that antenatal corticosteroids halved the risks of respiratory distress syndrome and mortality. It also confirmed that antenatal corticosteroids were effective in subgroups of patients in which the evidence from individual trials had been inconclusive. This meta-analysis may have played an important role in persuading obstetricians to change their practice3.

Disadvantages of meta-analysis

  1. Top of page
  2. The contribution of meta-analysis
  3. Disadvantages of meta-analysis
  4. Meta-analysis as a stimulus to larger trials
  5. How can we achieve more powerful trials?
  6. Conclusions
  7. References

Unfortunately most meta-analyses are themselves too small to provide reliable answers about mortality or major morbidity. For example, of three overviews of randomised trials of treatments in women at risk of preterm delivery in the Cochrane Database of Systematic Reviews4–6, two included less than 1500 patients per meta-analysis5,6. Even if a treatment reduced mortality or major complications by as much as a quarter, for example from 20% to 15%, ensuring more than 80% power to demonstrate this treatment effect would require 2000 patients. Even greater numbers are needed to demonstrate more moderate effects. As most of the overviews in perinatal medicine contain fewer patients than this, the impact of most treatments on mortality and major morbidity has yet to be reliably assessed.

Meta-analyses which are too small to give conclusive answers about mortality or major morbidity may actually slow progress, for at least four reasons. First, by providing conclusive evidence about less important, surrogate measures of outcome they may discourage clinicians from undertaking any further randomised trials. For example, as it is already clear from metaanalyses of randomised trials in women with preterm prelabour rupture of the membranes (PPROM) that antibiotics can reduce infections in mother and infant and prolong pregnancy5, some might conclude that antibiotics should routinely be used and that further trials would be unethical. However, the effect of antibiotics on neonatal mortality or later disability is still unknown. If the release of cytokines such as tumour necrosis factor in PPROM were to damage fetal brain before birth7,8, then prolonging pregnancy might increase the prevalence of long term disability. This possibility is underlined by two recent case control studies9,10. Both demonstrated substantial increases in disability among survivors born after PPROM versus those born after the onset of preterm labour with intact membranes, having adjusting for gestation. Prolonging the duration of PPROM with antibiotics or other agents might thus have net adverse effects on later disability. Similarly, a meta-analysis of trials in women in suspected preterm labour shows that indomethacin may significantly increase gestation at delivery and reduce the risk of intraventricular haemorrhage11. However, gestation at delivery and intraventricular haemorrhage are only intermediate or surrogate measures of outcome. As indomethacin is a potent vasoconstrictor of the fetal cerebral circulation it could cause profound cerebral ischaemia, leading to worse brain damage and excess disability. Such issues can only be resolved if women continue to participate in trials likely to be powerful enough to compare differences in substantive outcomes reliably, for example the ORACLE and the Antenatal TRH trials, provided such studies measure long term outcomes in survivors.

Second, meta-analyses based on inadequate numbers of patients may be misleading or controversial. For example, despite a meta-analysis of randomised trials in about 1300 patients with suspected myocardial infarction which indicated that mortality was substantially reduced by magnesium sulphate, a subsequent multicentre trial in over 58,000 patients showed that magnesium had no effect on mortality. It was suggested that the earlier meta-analysis may have been based on too few patients and deaths to allow reliable conclusions12. Researchers have begun to explore the degree of agreement between the results of large trials and meta-analyses of smaller trials and the reasons for disagreement in the minority of instances when it occurs13.

A third disadvantage of meta-analyses based on small trials is that evidence is inevitably accumulated more slowly than by collaborative studies of adequate size and power. Although the meta-analysis of the trials of antenatal corticosteroids2 is likely to have accelerated their use in women at risk of preterm delivery, it appeared nearly 20 years after the first work was published. Earlier confirmation of those findings in a larger collaborative trial might have had more rapid impact on practice. Fourth, combining trials with major differences in women's level of risk or severity of illness, such as the trials of aspirin in the prophylaxis of preeclampsia, may lead to differences in interpretation14,15. However, this problem also reflects the need for individual trials of adequate power. The controversy as to whether aspirin is effective in women at high risk of pre-eclampsia might never have arisen had early trials been sufficiently powerful to be definitive.

Meta-analysis as a stimulus to larger trials

  1. Top of page
  2. The contribution of meta-analysis
  3. Disadvantages of meta-analysis
  4. Meta-analysis as a stimulus to larger trials
  5. How can we achieve more powerful trials?
  6. Conclusions
  7. References

Despite their limitations, small trials are essential. By testing hypotheses generated in the laboratory or from the bedside, they facilitate advances in clinical practice. In some rare conditions, they may be the only way to obtain any unbiased evidence16. Small trials generate hypotheses about the relative effectiveness of competing treatments and can assist in prioritising the need for further trials. However, unless subsequent trials are powerful enough to provide reliable information on substantive outcomes such as mortality or major morbidity, patients may continue to be denied the most effective care.

How can we achieve more powerful trials?

  1. Top of page
  2. The contribution of meta-analysis
  3. Disadvantages of meta-analysis
  4. Meta-analysis as a stimulus to larger trials
  5. How can we achieve more powerful trials?
  6. Conclusions
  7. References

The practical aspects of running collaborative multicentre trials have been clearly reviewed by Warlow17. Many successful collaborative trials share basic principles of design and organisation, which, if widely applied, could substantially increase the size and power of randomised trials in perinatal medicine. Perhaps the most important principle is to keep the trial as simple as possible, so that a wide range of busy hospitals may realistically take part. This requires a ruthless commitment to exclude all items of data which are not relevant to the aims of the trial. A major benefit is that this makes it more feasible to collect complete and reliable data.

Conclusions

  1. Top of page
  2. The contribution of meta-analysis
  3. Disadvantages of meta-analysis
  4. Meta-analysis as a stimulus to larger trials
  5. How can we achieve more powerful trials?
  6. Conclusions
  7. References

Despite the clear need for trials of adequate size and power, too few are published in obstetrics and neonatal medicine. Clinicians need greater access to advice from colleagues with experience of conducting collaborative multicentre studies, and more opportunities for training in their design and organisation. Perhaps academic departments of perinatal or neonatal medicine should appoint their own epidemiologists and biostatisticians. This Journal has established a set of minimum standards for the reporting of randomised trials submitted for publication, which includes the description of a calculation of the sample size required for adequate statistical power1. Professionals, ethics committees, consumers and the media could help to enhance the power of future studies by encouraging a climate in which it is normal for pregnant women and their partners to be invited to participate in randomised trials, after receiving clear and sensitively presented information.

References

  1. Top of page
  2. The contribution of meta-analysis
  3. Disadvantages of meta-analysis
  4. Meta-analysis as a stimulus to larger trials
  5. How can we achieve more powerful trials?
  6. Conclusions
  7. References
  • 1
    Grant JM. Randomised trials and the British Journal of Obstetrics and Gynaecology. Minimum requirements for publication. Br J Obstet Gynaecol 1995; 102: 849850.
  • 2
    Crowley P, Chalmers I, Keirse MJNC. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. Br J Obstet Gynaecol 1990; 97: 1125.
  • 3
    Scottish Neonatal Consultants Collaborative Study Group and International Neonatal Network. Trends and variations in use of antenatal corticosteroids to prevent neonatal respiratory distress syndrome: recommendations for national and international comparative audit. Br J Obstet Gynaecol 1996; 103: 534540.
  • 4
    Crowley P. Corticosteroids prior to preterm delivery. In EnkinMW, KeirseMJNC, RenfrewMJ, NeilsonJP, editors. Pregnancy and childbirth module [computer program]. Cochrane Database of Systematic Reviews; 1994 May. Oxford : Cochrane Updates on Disk, Update Software, 1995. Disk Issue 2.
  • 5
    Crowley P. Antibiotics for preterm prelabour rupture of the membranes. In: EnkinMW, KeirseMJNC, RenfrewMJ, NeilsonJP, editors. Pregnancy and childbirth module [computer program]. Cochrane Database of Systematic Reviews; 1994 May. Oxford : Cochrane Updates on Disk, Update Software, 1995. Disk Issue 2.
  • 6
    Crowley P. Antibiotics in preterm labour with intact membranes. In: EnkinMW, KeirseMJNC, RenfrewMJ, NeilsonJP, editors. Pregnancy and childbirth module [computer program] of the Cochrane Database of Systematic Reviews; 1994 May. Oxford : Cochrane Updates on Disk, Update Software, 1993. Disk Issue 1.
  • 7
    Adinolfi M. Infectious diseases in pregnancy, cytokines and neuro- logical impairment: an hypothesis. Dev Med Child Neurol 1993; 35: 549553.
  • 8
    Leviton A. Preterm birth and cerebral palsy: Is tumor necrosis factor the missing link Dev Med Child Neurol 1993; 35: 553558.
  • 9
    Murphy DJ, Sellers S, MacKenzie IZ, Yudkin PL, Johnson AM. Case-control study of antenatal and intrapartum risk factors for cerebral palsy in very preterm singleton babies. Lancet 1995; 346: 14491454.
  • 10
    Spinillo A, Capuzzo E, Stronati M, Ometto A, Orcesi S, Fazzi E. Effect of preterm premature rupture of membranes on neurodevelopmental outcome: follow up at two years of age. Br J Obstet Gynaecol 1995; 102: 882887.
  • 11
    Keirse MJNC. indomethacin tocolysis in preterm labour. In: KeirseMJNC, RenfrewMJ, NeilsonJP, CrowtherC, editors. Pregnancy and Childbirth Module of The Cochrane Pregnancy and Childbirth Database. The Cochrane Collaboration; issue 2. Oxford : Update Software, 1995.
  • 12
    Egger M, Davey Smith G. Misleading meta-analysis. BMJ 1995; 310: 752754.
  • 13
    Cappelleri JC, Lonnidis JPA, Schmid CH, de Ferranti SD, Aubert MA, Chalmers TC, Lau J. Large trials vs meta-analysis of smaller trials. How do their results compare JAMA 1996; 276: 13321338.
  • 14
    Broughton Pipkin F, Crowther C, de Swiet M, Duley L, Judd A, Lilford RJ, Onwude J, Prentice C, Redman CWG, Roberts J, Thornton J, Walker J. Where next for prophylaxis against pre-eclampsia Br J Obstet Gynaecol 1996; 103: 603607.
  • 15
    Grant JM. Multicentre trials in obstetrics and gynaecology. Br J Obstet Gynaecol 1996; 103: 599602.
  • 16
    Lilford RJ, Thornton JG, Braunholtz D. Clinical trials and rare diseases: a way out of a conundrum. BMJ 1995; 311: 16211625.
  • 17
    Warlow C. How to do it. Organise a multicentre trial. BMJ 1990; 300: 180183.