Methodology Review

Publication bias in clinical trials due to statistical significance or direction of trial results

  1. Sally Hopewell1,*,
  2. Kirsty Loudon1,
  3. Mike J Clarke1,
  4. Andrew D Oxman2,
  5. Kay Dickersin3

Editorial Group: Cochrane Methodology Review Group

Published Online: 21 JAN 2009

Assessed as up-to-date: 23 OCT 2008

DOI: 10.1002/14651858.MR000006.pub3


How to Cite

Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews 2009, Issue 1. Art. No.: MR000006. DOI: 10.1002/14651858.MR000006.pub3.

Author Information

  1. 1

    UK Cochrane Centre, Oxford, Oxfordshire, UK

  2. 2

    Norwegian Knowledge Centre for Health Services, Oslo, Norway

  3. 3

    Johns Hopkins University, Center for Clinical Trials and US Cochrane Center, Baltimore, MD, USA

*Sally Hopewell, UK Cochrane Centre, National Institute for Health Research, Summertown Pavilion, Middle Way, Oxford, Oxfordshire, OX2 7LG, UK. shopewell@cochrane.co.uk.

Publication History

  1. Publication Status: New
  2. Published Online: 21 JAN 2009

SEARCH

 

Abstract

  1. Top of page
  2. Abstract
  3. Plain language summary
  4. 摘要

Background

The tendency for authors to submit, and of journals to accept, manuscripts for publication based on the direction or strength of the study findings has been termed publication bias.

Objectives

To assess the extent to which publication of a cohort of clinical trials is influenced by the statistical significance, perceived importance, or direction of their results.

Search methods

We searched the Cochrane Methodology Register (The Cochrane Library [Online] Issue 2, 2007), MEDLINE (1950 to March Week 2 2007), EMBASE (1980 to Week 11 2007) and Ovid MEDLINE In-Process & Other Non-Indexed Citations (March 21 2007). We also searched the Science Citation Index (April 2007), checked reference lists of relevant articles and contacted researchers to identify additional studies.

Selection criteria

Studies containing analyses of the association between publication and the statistical significance or direction of the results (trial findings), for a cohort of registered clinical trials.

Data collection and analysis

Two authors independently extracted data. We classified findings as either positive (defined as results classified by the investigators as statistically significant (P < 0.05), or perceived as striking or important, or showing a positive direction of effect) or negative (findings that were not statistically significant (P ≥ 0.05), or perceived as unimportant, or showing a negative or null direction in effect). We extracted information on other potential risk factors for failure to publish, when these data were available.

Main results

Five studies were included. Trials with positive findings were more likely to be published than trials with negative or null findings (odds ratio 3.90; 95% confidence interval 2.68 to 5.68). This corresponds to a risk ratio of 1.78 (95% CI 1.58 to 1.95), assuming that 41% of negative trials are published (the median among the included studies, range = 11% to 85%). In absolute terms, this means that if 41% of negative trials are published, we would expect that 73% of positive trials would be published.

Two studies assessed time to publication and showed that trials with positive findings tended to be published after four to five years compared to those with negative findings, which were published after six to eight years. Three studies found no statistically significant association between sample size and publication. One study found no significant association between either funding mechanism, investigator rank, or sex and publication.

Authors' conclusions

Trials with positive findings are published more often, and more quickly, than trials with negative findings.

 

Plain language summary

  1. Top of page
  2. Abstract
  3. Plain language summary
  4. 摘要

Publication bias in clinical trials due to statistical significance or direction of trial results

The validity of a systematic review depends on the methods used to conduct the review. If there is a systematic bias, such that studies with statistically significant or positive findings are more likely to be published and included in systematic reviews than trials with non-significant findings, then the validity of a review's conclusions can be threatened.

This methodology review identified five studies that investigated the extent to which the publication of clinical trials (such as those approved by an ethics review board) is influenced by the statistical significance or direction of a trial's results. These studies showed that trials with positive findings (defined either as those that were statistically significant (P < 0.05), or those findings perceived to be important or striking, or those indicating a positive direction of treatment effect), had nearly four times the odds of being published compared to findings that were not statistically significant (P ≥ 0.05), or perceived as unimportant, or showing a negative or null direction of treatment effect. This corresponds to a risk ratio of 1.78 (95% CI 1.58 to 1.95), assuming that 41% of negative trials are published.Two studies found that trials with positive findings also tended to be published more quickly than trials with negative findings. The size of the trial (assessed in three studies) and the source of funding, academic rank, and sex of the principal investigator (assessed in one study) did not appear to influence whether a trial was published.

These results provide support for mandating that clinical trials are registered before recruiting participants so that review authors know about all potentially eligible studies, regardless of their findings. Those carrying out systematic reviews should ensure they assess the potential problems of publication bias in their review and consider methods for addressing this issue by ensuring a comprehensive search for both published and unpublished trials.

 

摘要

  1. Top of page
  2. Abstract
  3. Plain language summary
  4. 摘要

背景

統計顯著性或試驗結果的方向性造成臨床試驗的發表偏差(Publication bias)

作者們是否將研究結果投稿、雜誌編輯們是否接受投稿﹐會受到研究結果的方向或強度所影響﹐被稱做發表偏差。

目標

評估同一世代的臨床試驗結果是否發表受到1) 研究結果的統計顯著性、2) 被認知的重要性或3) 結果的方向性影響的程度。

搜尋策略

搜尋Cochrane Methodology Register (The Cochrane Library [Online] 2007年第2期), MEDLINE (1950 至 2007年3月第2週), EMBASE (1980 至2007年第11週) 及Ovid MEDLINE InProcess& Other NonIndexed Citations(2007年3月21日)。也搜尋了Science Citation Index (2007年4月),檢查相關文章之參考文獻清單﹐並與研究人員聯絡﹐找出其他研究。

選擇標準

分析某一時期登記的所有臨床試驗,其結果的發表與否,與其結果的統計顯著性或方向性之間的關連性的研究。

資料收集與分析

兩位作者各自擷取資料。我們把結果區分為陽性(定義為研究者認為結果具統計上顯著意義(P < 0.05),或認知為重要的或重大的,或效果呈現正向的)或陰性(結果不具統計上顯著意義(P ? 0.05), 或認知為不重要的,或效果呈現負向的或無效)。如果相關資料可以取得,同時擷取可能導致結果不被發表的因子。

主要結論

共納入5個研究。具陽性結果的臨床試驗比陰性者更可能被發表(odds ratio 3.90; 95% confidence interval 2.68 to 5.68)。假設陰性結果的試驗有41%的機會被發表(中位數,納入的研究範圍從11% 至 85%不等)﹐相當於風險比(risk ratio)1.78 (95% CI 1.58 to 1.95)。以絕對比例而言﹐這意味著如果陰性結果的研究有41%被發表﹐則陽性結果的研究有73%被發表。有兩個研究評估發表所需的時間﹐結果顯示,具陽性結果者在4至5年後發表,而陰性結果者在6至8年後發表。有3篇研究樣本數和是否發表沒有顯著關連。一個研究發現研究經費來源、研究者職級或性別和是否發表無顯著關連。

作者結論

具陽性結果之臨床試驗比陰性結果者更常、更快被發表。

翻譯人

本摘要由慈濟醫院葉日弌翻譯。

此翻譯計畫由臺灣國家衛生研究院(National Health Research Institutes, Taiwan)統籌。

總結

系統性回顧的效度決定於完成回顧的方法。如果有系統性的誤差,例如具統計上顯著意義、或效果呈現正向的研究﹐比不具統計上顯著意義的研究容易被發表、且容易被納入在系統性回顧中,則系統性回顧得到的結論正確性可能有問題。本篇方法學回顧納入5篇﹐評估臨床試驗結果發表與否受到研究結果的統計顯著性、或方向性影響的程度。研究顯示﹐具陽性(定義為研究者認定結果具統計上顯著意義(P < 0.05),或認知為重要的或重大的,或效果呈現正向的)結果的臨床試驗被發表的勝率﹐幾乎是臨床試驗結果不具統計上顯著意義(P ? 0.05)、或認知為不重要的,或效果呈現負向的或無效者的4倍。假設陰性結果的試驗有41%的機會被發表,這相當於風險比為1.78 (95% CI 1.58 to 1.95)。有2個研究發現﹐具陽性結果之臨床試驗比具陰性結果者更常、更快被發表。臨床試驗的樣本數(有3個研究評估)、研究經費來源、研究者職級或性別(有1個研究評估)和是否發表無顯著關連。這些結果支持以下作法﹕臨床試驗在招募受試者之前必須登記﹐以便無論結果為何,文獻回顧者能囊括所有可能的相關研究。進行系統性回顧者應確實評估發表偏差的潛在問題,並考慮對已發表及未發表的臨床試驗均做完整的搜尋﹐確保解決此問題。