The effect of the publication of a major clinical trial in a high impact journal on clinical practise: the ORACLE Trial experience
*Correspondence: S. Kenyon, Obstetrics and Gynaecology, Robert Kilpatrick Clinical Sciences Building, Leicester Royal Infirmary, P.O. Box 65, Leicester LE2 7LX, UK.
Objective To estimate the short term effect of the publication of a major clinical trial on clinical practise.
Design Questionnaire survey of clinical practise.
Population All maternity units in the UK.
Method A self-administered questionnaire completed by lead consultants on delivery suite of maternity units.
Main outcome measures Changes in antibiotic prescription.
Results Within six months of publication, approximately 50% of maternity units had changed their guidelines for the care of women with preterm prelabour rupture of the fetal membranes.
Conclusion Publication of a major clinical trial does impact on clinical practise but the impact is heterogeneous in terms of time and consistency.
There is widespread recognition of the need to improve the effectiveness of the implementation of research findings into clinical practise because of the continuing gap between research findings and clinical practise and the need to demonstrate that public investment in research results in better patient care and outcomes. In order to promote the uptake of research findings, it is necessary to identify the potential barriers to implementation and to develop strategies to overcome them. Specific interventions that have been used to promote research-based practise include using clinical guidelines, computerised decision support systems, educational programmes, communicating research findings to clinicians and patients and developing strategies for organisational change1. We report here the short term effect of the publication of a major trial results in a high impact clinical journal.
This study reports the impact on reported clinical practise of the publication of a MRC funded, large, international pragmatic perinatal randomised controlled trial—ORACLE. The trial tested the hypothesis that the treatment of women under 37 weeks of gestation in either spontaneous preterm labour (SPL) or with preterm prelabour ruptured membranes (pPROM) with broad spectrum antibiotics (co-amoxiclav and erythromycin) prolonged pregnancy and improved neonatal mortality and morbidity. The trial randomised 11,155 women from 161 maternity units (135 within the UK). The trial began in July 1994 and completed recruitment in May 2000. The results were published in the Lancet in March 20012,3.
A total of 6295 women with spontaneous preterm labour were randomly assigned to 250 mg erythromycin, 325 mg co-amoxiclav, both antibiotics or neither. The results showed that neither of the antibiotics was associated with any improvement in neonatal mortality or morbidity.
A total of 4826 women with preterm prelabour ruptured membranes were randomly assigned the same treatments. Results showed that the prescription of erythromycin to women with singleton pregnancies resulted in statistically significant reductions in composite primary outcome (neonatal death, chronic lung disease, abnormal cerebral ultrasound scans prior to discharge from hospital). Both antibiotics prolonged pregnancy but the prescription of erythromycin to women was associated reductions in neonatal morbidity. An unexpected finding was that the prescription of co-amoxiclav was associated with an increase in neonatal necrotising enterocolitis.
In September 2001, six months after publication of trial results, a self-administered questionnaire was sent to the lead consultant for delivery suite in maternity units with 24 hour obstetric service in England, Wales, Scotland and Northern Ireland to ascertain the short term impact of the ORACLE Trial results. A reminder was sent three weeks later. One hundred forty-nine responses were received. The primary author then phoned the delivery suite in a further 43 units to ascertain the relevant information from the midwife in charge. The questionnaire asked about practise before and after the publication of ORACLE results, about prescription of co-amoxiclav on delivery suite, whether the person completing the questionnaire was aware of the publication of the ORACLE results and whether they had read them.
The questionnaire was sent to 252 units; in total 192 responded (i.e. a 76% response rate).
One hundred seventy-six (92%) reported that they were aware of the publication and 154 (80%) reported that they had read it.
Of the units to respond, 123 (64%) had collaborated and 69 (36%) had not.
The vast majority of maternity units (163) reported they do not prescribe antibiotics for women in spontaneous preterm labour. However, 12 started to prescribe erythromycin and three had begun prescription of another antibiotic. One unit had stopped prescribing co-amoxiclav leaving seven continuing to do so (see Table 1).
Table 1. Impact on antibiotic prescription for women in spontaneous preterm labour. ‘Before’ and ‘After’ values are given as n (%).
|No antibiotic prescription||177 (92)||163 (85)||−7|
|Erythromycin prescription||3 (2)||15 (8)||+6|
|Co-amoxiclav prescription||8 (4)||7 (3.5)||−0.5|
|Other antibiotic prescription||4 (2)||7 (3.5)||+1.5|
The number of maternity units not prescribing antibiotics to women with preterm prelabour ruptured membranes reduced from 147 (77%) to 78 (41%) after the trial results. Ninety-two units changed to prescribing erythromycin, an increase of 46%. Twelve units (6%) prescribed co-amoxiclav (a reduction of 15) and 10 units prescribed another antibiotic following publication of the trial results (see Table 2).
Table 2. Impact on antibiotic prescription for women in preterm prelabour ruptured membranes. ‘Before’ and ‘After’ values are given as n (%).
|No antibiotic prescription||147 (77)||78 (41)||−36|
|Erythromycin prescription||4 (2)||92 (48)||+46|
|Co-amoxiclav prescription||27 (14)||12 (6)||−8|
|Other antibiotic prescription||14 (7)||10 (5)||−2|
In the questionnaire, we also asked whether there had been any change in the prescription of co-amoxiclav in maternity units. It would appear that since the ORACLE Trial results were published, there had been a reduction in the use of co-amoxiclav during antenatal and intrapartum period. Seventy-one units (37%) had stopped prescription, 89 (46%) reported they were continuing it and 31 units continued not to use co-amoxiclav (see Table 3).
Table 3. Reported use of co-amoxiclav on delivery suite. Values are given as n (%).
|Continued to use co-amoxiclav||89 (46.4)|
|Stopped using co-amoxiclav||71 (37)|
|Started to use co-amoxiclav||1 (0.5)|
|Continued not to use co-amoxiclav||31 (16.1)|
Historically, practise responds slowly to research findings. An early example of delayed uptake of innovations was the use of lemon juice to prevent scurvy. In 1601, James Lancaster showed that lemon juice was effective but it was not until 1747 that James Lind confirmed the results and then the British Navy did not fully adopt this innovation until 1795 (not until 1865 in the case of Merchant Marines)4. Among more recent examples of delayed implementation of research results are the ascertainment and control for hypertension5,6, the inadequate use of prophylactic antibiotics in patients having orthopaedic surgery7, the inadequate treatment of asthma8 and the inadequate treatment of children with gastroenteritis9. In the case of thrombolytic treatment for myocardial infarction, there was a 13-year delay between the demonstration of effectiveness from cumulative meta-analysis of randomised controlled trials and the advocacy of it by most authors of review articles or book chapters10.
Recent perinatal examples of the impact of evidence on practise report similar delays in the uptake of findings by clinicians. The administration of corticosteroids to mothers expected to deliver prematurely reduces neonatal mortality and morbidity11. The first trial which suggested that corticosteroids were effective in this role was published in 1972 and evidence from 12 trials was assembled in a systemic review published in January 199011. Despite the accumulating evidence in 1991, many women delivering prematurely in the UK and elsewhere were not receiving corticosteroids12,13. The delay in changing practise may also have been in part due to the fact that the treatment needed to be administered by the obstetricians but the benefits in improved outcomes were seen by the neonatologist and paediatricians as well as the parents: a case similar to the MRC ORACLE Trial.
It may well also be that the speed at which research findings are implemented is related to the size of the benefit demonstrated. The Eclampsia Trial results14, published in 1995, provided strong evidence that magnesium sulphate was of benefit for women with eclampsia. A survey of practise15, published in 1998, showed that 60% of maternity units who responded were using magnesium sulphate for women with eclampsia. The relatively small benefit seen with the prescription of erythromycin to women with preterm prelabour ruptured membranes demonstrated by the MRC ORACLE Trial results may have triggered considerable debate in maternity units about the strength of evidence, and it is possible that this has contributed to delayed implementation. There are also many local hurdles to implementing guidelines and delivery unit guidelines groups have many pressures and proposals thrust on them.
It is interesting to note that a considerable number of units opted for a policy which was not consistent with the results of the MRC ORACLE Trial. For women in spontaneous preterm labour, 16 units started prescribing antibiotics. For women with preterm prelabour ruptured membranes, while 88 units began prescribing erythromycin, 78 continued not to prescribe antibiotics and 12 continued to prescribe co-amoxiclav. This demonstrates differing interpretations among clinicians of the MRC ORACLE Trial results.
It is also interesting to note that the unexpected finding of increased neonatal necrotising enterocolitis associated with the prescription of co-amoxiclav has resulted in a reduction in the numbers if units using this antibiotic during the antenatal and intrapartum period.
It would seem that researchers do not generally measure the impact of their research. One example is the lack of reference to the impact of the research regarding the Term PROM Study16. It is increasingly apparent that researcher's have a responsibility to assist the implementation to monitor the impact of this. Funders of research are increasingly interested to measure the impact of their research. Perhaps funders should require their researchers to submit proposals of how they will ensure implementation. In their turn, funders will have to provide resources for this.
There is evidence of the hard work, many dimensions and cost of implementing change in practise17. Here we report an important change in practise by nearly half the obstetricians who replied to our survey in the UK without any universal strategies being employed. Once again, this study demonstrates the complexity on implementing research findings.
The authors would like to thank the clinicians who returned the questionnaire, Ann Blackburn and Kate Taylor in the ORACLE office, Lelia Duley and Peter Brocklehurst who encouraged the idea and commented on the questionnaire.