How to reduce unnecessary duplication: use PROSPERO

Authors

  • D Moher,

    Corresponding author
    1. Faculty of Medicine, Clinical Epidemiology Program, Ottawa Hospital Research Institute, University of Ottawa, Ottawa, ON, Canada
    • Correspondence: D Moher, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital - General Campus, 501 Smyth Rd, Room L1288, Ottawa, ON K1H 8L6, Canada. Email dmoher@ohri.ca

    Search for more papers by this author
  • A Booth,

    1. NIHR Centre for Reviews and Dissemination, University of York, York, UK
    Search for more papers by this author
  • L Stewart

    1. NIHR Centre for Reviews and Dissemination, University of York, York, UK
    Search for more papers by this author

In this issue of BJOG two systematic reviews are published addressing the same question, namely, is chewing gum effective in reducing postoperative ileus for women undergoing a caesarean section.[1, 2] Both reviews produce the same result, at least for one main outcome, time to first flatus, although both groups may have used different methods to arrive at their answer. For example, the eligibility criteria, or how they were implemented, might have differed between the reviews as each review arrived at a different number of included studies. Although this is not necessarily an example of discordant reviews,[3] what will be obvious to most readers is a case of duplicate publication.[4] Whether these are examples of unnecessary duplication[5] can be assessed through a basic search of PubMed Health, The Cochrane Library and Medline, which identifies at least two published reviews covering the same intervention for abdominal surgery. These reviews were only referenced by Craciunas et al.[1]

Unfortunately neither of the reviews published here was registered on PROSPERO, an international prospective register for systematic review protocols.[6] As such, the two groups are unlikely to have been aware of each other's work, resulting in unplanned duplication. Similarly, neither report mentions the existence of an ongoing update of a review using the same intervention for intra-abdominal surgery, which was registered on PROSPERO in January 2013.[7] Neither review reports the existence of a protocol—an approach that deviates from best practice.[8, 9] Sadly they are not alone. A review of 175 non-Cochrane reviews published in November 2004 indicates that only 14% reported working from an existing protocol.[10] A more recent review examining systematic reviews published between January 2010 and May 2011 reported that <20% of them mentioned a protocol;[11] anything less than 100% is troubling. As for other types of research, systematic reviewers need to have a working plan for their proposed systematic review, which should include a clear description of inclusion and exclusion criteria and the outcomes that the review intends to address and how any analysis will be done. To avoid bias and the perception of bias the review should report clearly any deviation from the protocol.

In order that completed reviews can be compared with what was planned at their outset, PROSPERO only accepts prospective registration of planned reviews. Authors of completed reviews cannot back-fill and retrospectively seek ‘prospective registration’. Were they to attempt to manipulate the stage of their review to obtain registration, such falsification is unfortunate at the very least and of questionable ethics. It is important to seek registration in PROSPERO early on in the planning process, ideally before screening against eligibility criteria commences and certainly well before data extraction is completed. Systematic reviews of interventions and strategies to prevent, diagnose, treat and monitor health conditions can be registered as long as there is a health-related outcome. Systematic reviews of reviews, of qualitative studies, and of methodology also qualify as long as they meet the timing and outcome requirements.

Although registration was developed initially for reviews of interventions, the long-term aim is to include all systematic reviews with a health-related outcome in the broadest sense, already registrants are managing to tailor responses appropriately for a variety of non-intervention reviews and a pragmatic approach to inclusion in the register is being taken.

PROSPERO was launched in February 2011 before either of the new systematic reviews of chewing gum were started. At the time of writing more than 2200 protocols from 66 countries have been registered. PROSPERO appears to be gaining momentum. The average number of protocol registrations per month is now 133, a major increase in its trajectory compared with an average of 65 a month at the same time in 2012. No doubt enlightened policy-making, such as by the UK's National Institute of Health Research (NIHR), requiring recipients of NIHR funding to register their protocol on PROSPERO,[12] and journals such as this one,[13] has helped to implement registration. Later this year Cochrane protocols will be automatically added to PROSPERO.

Registering on PROSPERO is not difficult. There are only 18 mandatory items [and 22 optional ones]. The identification of items to include was based on established methods and involved a broad spectrum of stakeholders including systematic reviewers.[14] It takes about 30–60 minutes to register a review in the majority of cases, particularly where there is a well-written proposal or protocol. We can safely assume that completing a ‘typical’ systematic review takes 1000 person-hours; even assuming 60 minutes to register this task only accounts for 0.001% of the total time. The majority of systematic reviewers will probably find this time allocation very acceptable. The PROSPERO registration form, downloadable as an editable document, can also be used by researchers as a template to help develop their full protocol.

An important aim of PROSPERO is to, hopefully, reduce unplanned duplication. Waugh has posited the possibility of people going to the registry and identifying topics for which they can very quickly complete the review and publish it before the team who registered their review do.[15] While there may be some remote possibility of this happening, there is no evidence to support such a scenario. Good peer review will help to reduce whatever remote possibility exists. Part of any peer review of a systematic review should include the examination of a completed or published protocol and/or registration information. A search of PROSPERO to identify similar questions should alert peer reviewers to the possibility of unscrupulous behaviour. What is possibly more problematic is that some researchers may develop and complete reviews even when they know from the registration details that others are already working on the same topic. In jurisdictions for which publications are linked with financial reward we need to consider ways to decouple this link or at least to be aware of it.[16]

Teaching the next generation of systematic reviewers is another opportunity to help promote the good use of limited resources and the avoidance of inappropriate duplicate reviews. It is good scientific practice to identify existing reviews, either protocols or completed reviews, before jumping in and conducting a review. This should be integral to the review process.[8, 9] Following development of a clearly formulated question, systematic reviewers should search for existing systematic review protocols and completed reviews. PROSPERO is a good place to identify existing protocols and the Cochrane Library and Database of Abstracts of Reviews of Effects are excellent places to identify high-quality completed reviews. Open access publication is likely to facilitate reviewers more thoroughly examining identified protocols and/or completed reviews.

If existing protocols are identified the review team should review the eligibility criteria and outcomes to see whether they meet their interest or whether there are relevant nuances, such as different patient subgroups or outcomes that might warrant continuation of their additional review. Sometimes duplicate reviews are undertaken to satisfy sensitive political and jurisdictional needs. It is possible that the methodology of an existing review was weak and a new review is being conducted to overcome these deficiencies. Duplication of reviews can be appropriate, but should always be planned and a simple statement justifying planned duplication should be included in the final report.

There are many other ways in which PROSPERO can be useful. It can be used to carry out epidemiological investigations, such as examining the uptake of prospective registration across countries, so providing information about implementation. Similarly, having access to relevant protocol information allows researchers to examine the extent of potential reporting biases. Silagy et al.[17] documented an early example and more recent evidence points to reporting bias as a concern for the systematic review community.[18] Similar research using clinical trial registries has uncovered serious problems with outcome reporting bias.[19, 20] Some time ago Stern and Simes[21] documented the time lines between clinical trial registration and publication. Statistically significant trials were more likely to be published with so-called ‘negative trials’ taking twice as long. PROSPERO could be used to address similar questions regarding systematic reviews.

The PRISMA statement, guidance for reporting systematic reviews and meta-analyses[22] is currently being extended to protocols of these reviews in much the same way that SPIRIT was developed for protocols of randomised trials.[23] PRISMA-P (protocols) consists of an 17-item checklist and its development is based heavily on the work of PROSPERO.[24] This guidance is about what should, ideally, be included in a protocol of a systematic review. As such it is moving away from strict reporting guidance. In developing PRISMA-P we have also worked hard to ensure that checklist items are similar to what authors are asked to report in their protocol and parts of their completed review. We believe that systematic reviewers can efficiently register on PROSPERO and write-up their complete protocol and have a good deal of their methods already drafted for the publication submission of their completed review. All this, while contributing to a reduction in unplanned duplication and demonstrating good practice in review methods.

Disclosure of interest

All three authors are members of the PROSPERO advisory group. AB is a research fellow at the NIHR Centre for Reviews and Dissemination and is responsible for the development and maintenance of PROSPERO.

Contribution to authorship

DM wrote the initial draft after which all authors contributed equally in the revisions.

Funding

No funding was received.

Acknowledgements

DM is supported by a University Research Chair. LS is funded through the NIHR and is Director of CRD, which initiated and manages PROSPERO, being also part of its advisory group.