Methodology Review

Editorial peer review for improving the quality of reports of biomedical studies

  1. Tom Jefferson1,*,
  2. Melanie Rudin2,
  3. Suzanne Brodney Folse3,
  4. Frank Davidoff4

Editorial Group: Cochrane Methodology Review Group

Published Online: 18 APR 2007

Assessed as up-to-date: 19 FEB 2007

DOI: 10.1002/14651858.MR000016.pub3


How to Cite

Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database of Systematic Reviews 2007, Issue 2. Art. No.: MR000016. DOI: 10.1002/14651858.MR000016.pub3.

Author Information

  1. 1

    The Cochrane Collaboration, Vaccines Field, Roma, Italy

  2. 2

    Health Reviews Ltd, Roma, Italy

  3. 3

    Blue Cross Blue Shield of Rhode Island, Health and Wellness Division, Providence, Rhode Island, USA

  4. 4

    Institute for Healthcare Improvement, Hartford, CT 06109, USA

*Tom Jefferson, Vaccines Field, The Cochrane Collaboration, Via Adige 28a, Anguillara Sabazia, Roma, 00061, Italy. jefferson.tom@gmail.com. jefferson@assr.it; jefferson.tom@gmail.com.

Publication History

  1. Publication Status: Edited (no change to conclusions)
  2. Published Online: 18 APR 2007

SEARCH

 

Abstract

  1. Top of page
  2. Abstract
  3. Plain language summary
  4. 摘要

Background

Scientific findings must withstand critical review if they are to be accepted as valid, and editorial peer review (critique, effort to disprove) is an essential element of the scientific process. We review the evidence of the editorial peer-review process of original research studies submitted for paper or electronic publication in biomedical journals.

Objectives

To estimate the effect of processes in editorial peer review.

Search methods

The following databases were searched to June 2004: CINAHL, Ovid, Cochrane Methodology Register, Dissertation abstracts, EMBASE, Evidence Based Medicine Reviews: ACP Journal Club, MEDLINE, PsycINFO, PubMed.

Selection criteria

We included prospective or retrospective comparative studies with two or more comparison groups, generated by random or other appropriate methods, and reporting original research, regardless of publication status. We hoped to find studies identifying good submissions on the basis of: importance of the topic dealt with, relevance of the topic to the journal, usefulness of the topic, soundness of methods, soundness of ethics, completeness and accuracy of reporting.

Data collection and analysis

Because of the diversity of study questions, viewpoints, methods, and outcomes, we carried out a descriptive review of included studies grouping them by broad study question.

Main results

We included 28 studies. We found no clear-cut evidence of effect of the well-researched practice of reviewer and/or author concealment on the outcome of the quality assessment process (9 studies). Checklists and other standardisation media have some evidence to support their use (2 studies). There is no evidence that referees' training has any effect on the quality of the outcome (1 study). Different methods of communicating with reviewers and means of dissemination do not appear to have an effect on quality (3 studies). On the basis of one study, little can be said about the ability of the peer-review process to detect bias against unconventional drugs. Validity of peer review was tested by only one small study in a specialist area. Editorial peer review appears to make papers more readable and improve the general quality of reporting (2 studies), but the evidence for this has very limited generalisability.

Authors' conclusions

At present, little empirical evidence is available to support the use of editorial peer review as a mechanism to ensure quality of biomedical research. However, the methodological problems in studying peer review are many and complex. At present, the absence of evidence on efficacy and effectiveness cannot be interpreted as evidence of their absence. A large, well-funded programme of research on the effects of editorial peer review should be urgently launched.

 

Plain language summary

  1. Top of page
  2. Abstract
  3. Plain language summary
  4. 摘要

Editorial peer review for improving the quality of reports of biomedical studies

Editorial peer review is used world-wide as a tool to assess and improve the quality of submissions to paper and electronic biomedical journals. As the information revolution gathers pace, an empirically proven method of quality assurance is of paramount importance. The increasing availability of empirical research on the possible effects of peer review led us to carry out a review of current evidence on the efficacy of editorial peer review. We found few studies of reasonable quality, and most of these were concerned with the effects of blinding reviewers and/or authors to each others' identity. We could not identify any methodologically convincing studies assessing the core effects of peer review. Major research is urgently needed.

 

摘要

  1. Top of page
  2. Abstract
  3. Plain language summary
  4. 摘要

背景

同儕審查機制改善生物醫學期刊品質之效果

科學的發現必須經過批判性審查,才能被認可。而同儕審查(批判,設法反證)是科學過程的必要元素。我們回顧投稿紙本或電子生物醫學期刊同儕審查機制的實證基礎。

目標

估計各種同儕審查方式的效果。

搜尋策略

搜尋至2004年6月為止的包含下之列資料庫CINAHL, Ovid, Cochrane Methodology Register, Dissertation abstracts, EMBASE, Evidence Based Medicine Reviews: ACPJournal Club, MEDLINE, PsycINFO, PubMed。

選擇標準

本回顧包含前瞻或回溯性比較﹐以隨機或其他適當方式產生的兩個以上群體的原創性研究,不論其出版的狀態為何。我們希望找出以下列標準認定好的的研究:處理重要的主題,主題對雜誌的適切性,主題的實用性,方法扎實,倫理基礎穩固,報告既完整又精確。

資料收集與分析

因為研究提問、觀點、方法及結果的分歧,將納入的研究提問簡單分組,作描述性的回顧。

主要結論

我們總共納入28個研究。隱匿審查者及作者姓名的常規作法﹐對最後的品質影響,缺乏明確的證據(9個研究)。有些證據支持使用檢查清單及標準化媒介(2個研究)。訓練審查者對結果品質的影響﹐也沒有證據支持(1個研究)。與審查者溝通的方式及散佈資訊的方法沒有效果(3個研究)。某個研究指出,同儕審查對非傳統藥物偏見的審查檢視能力﹐我們所知甚少。同儕審查的效度只在某專門領域有個小規模的測試。編輯的同儕審查機制似乎提高論文的可讀性,並改善整份報告的品質(2個研究),但這方面的證據在不同條件下的應用性很有限。

作者結論

目前仍少有實證證據支持以同儕審查確保生物醫學研究品質的機制。然而研究同儕審查問題,在方法學上的問題很多且複雜。目前缺少有效性的支持證據﹐不能解釋為此機制無效。應該要儘速展開大規模,經費充裕的計畫來研究同儕審查的效果。

翻譯人

本摘要由慈濟醫院葉日弌翻譯。

此翻譯計畫由臺灣國家衛生研究院(National Health Research Institutes, Taiwan)統籌。

總結

同儕審查是全球評估及改善生物醫學紙本及電子版雜誌稿件的工具。隨著資訊革命的腳步加快,以實證基礎的方法來確保品質益發重要。有關同儕審查效果的實證研究逐漸增多,讓我們可以回顧這一主題的證據。我們發現極少研究有合理的品質,且絕大部分在探討隱匿審查者及作者姓名的效果。在評估同儕審查的核心效果方面﹐我們找不到任何在方法學上具說服力的研究,對此亟需更多研究。