Industry sponsorship and research outcome

  • Comment
  • Review
  • Methodology

Authors


Abstract

Background

Clinical research affecting how doctors practice medicine is increasingly sponsored by companies that make drugs and medical devices. Previous systematic reviews have found that pharmaceutical industry sponsored studies are more often favorable to the sponsor’s product compared with studies with other sources of sponsorship. This review is an update using more stringent methodology and also investigating sponsorship of device studies.

Objectives

To investigate whether industry sponsored drug and device studies have more favorable outcomes and differ in risk of bias, compared with studies having other sources of sponsorship.

Search methods

We searched MEDLINE (1948 to September 2010), EMBASE (1980 to September 2010), the Cochrane Methodology Register (Issue 4, 2010) and Web of Science (August 2011). In addition, we searched reference lists of included papers, previous systematic reviews and author files.

Selection criteria

Cross-sectional studies, cohort studies, systematic reviews and meta-analyses that quantitatively compared primary research studies of drugs or medical devices sponsored by industry with studies with other sources of sponsorship. We had no language restrictions.

Data collection and analysis

Two assessors identified potentially relevant papers, and a decision about final inclusion was made by all authors. Two assessors extracted data, and we contacted authors of included papers for additional unpublished data. Outcomes included favorable results, favorable conclusions, effect size, risk of bias and whether the conclusions agreed with the study results. Two assessors assessed risk of bias of included papers. We calculated pooled risk ratios (RR) for dichotomous data (with 95% confidence intervals).

Main results

Forty-eight papers were included. Industry sponsored studies more often had favorable efficacy results, risk ratio (RR): 1.32 (95% confidence interval (CI): 1.21 to 1.44), harms results RR: 1.87 (95% CI: 1.54 to 2.27) and conclusions RR: 1.31 (95% CI: 1.20 to 1.44) compared with non-industry sponsored studies. Ten papers reported on sponsorship and effect size, but could not be pooled due to differences in their reporting of data. The results were heterogeneous; five papers found larger effect sizes in industry sponsored studies compared with non-industry sponsored studies and five papers did not find a difference in effect size. Only two papers (including 120 device studies) reported separate data for devices and we did not find a difference between drug and device studies on the association between sponsorship and conclusions (test for interaction, P = 0.23). Comparing industry and non-industry sponsored studies, we did not find a difference in risk of bias from sequence generation, allocation concealment and follow-up. However, industry sponsored studies more often had low risk of bias from blinding, RR: 1.32 (95% CI: 1.05 to 1.65), compared with non-industry sponsored studies. In industry sponsored studies, there was less agreement between the results and the conclusions than in non-industry sponsored studies, RR: 0.84 (95% CI: 0.70 to 1.01).

Authors' conclusions

Sponsorship of drug and device studies by the manufacturing company leads to more favorable results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard 'Risk of bias' assessments.

초록

기업 후원이 연구 결과에 미치는 영향

배경

의사의 진료 방식에 영향을 미치는 임상연구에 대한 제약 및 의료기기 제조 기업의 후원이 점차 증가하고 있다. 이전 체계적 문헌고찰에서는 제약 기업 후원 임상연구가 그렇지 않은 연구에 비해 더 자주 후원 기업의 제품에 유리한 성향을 띤다는 점을 찾아내었다. 이 리뷰는 보다 엄밀한 방법론을 적용하였으며 약물 뿐 아니라 의료 기기 임상연구에 대한 기업후원에 대해서도 조사하였다.

목적

약물 및 의료기기 제조 기업의 후원을 받은 연구가 그렇지 않은 연구에 비해 시험 제품에 대해 보다 유리한 성향을 띠며 바이어스 위험 측면에서 차이가 나는지 조사하기 위함.

검색 전략

메드라인 (MEDLINE) (1948년부터 2010년 9월까지), 엠베이스 (EMBASE) (1980년부터 2010년 9월까지), 코크란 방법론 그룹 연구 등록부 (Cochrane Methodology Register) (2010년 4호), 웹 오브 사이언스 (Web of Science) (2011년 8월) 를 검색하였다. 더불어, 포함된 연구의 참고문헌 목록과 기존 체계적 문헌고찰 및 저자의 자료파일에서도 검색을 수행하였다.

선정 기준

단면 연구, 코호트 연구, 체계적 문헌고찰 및 메타 분석 등 기업 후원 약물/의료기기 일차 연구와 기업 후원을 받지 않은 일차 연구를 양적 방법론으로 비교한 연구를 분석대상으로 삼았다. 일차 연구는 어떤 언어로 보고되었던 상관없었다.

자료 수집 및 분석

평가자 두 명이, 연관되어 보이는 연구를 골라내었다. 분석 대상 연구의 최종 선택은 모든 저자가 함께 결정내렸다. 평가자 두 명이 자료를 추출하였다. 또한 우리는 분석에 포함된 연구의 저자들에게 연락하여 추가적인 미출간 자료가 있는지 물어보았다. 유리한 결과 보고, 유리한 결론 보고, 효과 크기, 바이어스 위험 및 연구 결과와 결론의 일치 여부를 이 분석의 결과 평가 항목으로 삼았다. 평가자 두 명이 분석에 포함된 연구의 바이어스 위험을 평가하였다. 이분형 결과에 대해서는 상대 위험도 (RR) 와 95% 신뢰구간을 계산하였다.

주요 결과

48건의 연구가 분석되었다. 기업 후원을 받은 연구는 그렇지 않은 연구에 비해 보다 자주 시험 약물/기기에 유리한 방향으로 효능 (RR 1.32, 95% CI 1.21 to 1.44), 위해 (RR 1.87, 95% CI 1.54 to 2.27), 결론 (RR 1.31, 95% CI 1.20 to 1.44) 을 보고하였다. 연구 10건은 기업 후원와 효과 크기에 대해 보고하였지만, 자료 보고 방식이 상이하여 통합 분석될 수 없었다. 연구들은 이질적이었다. 5개 연구에서는 기업 후원을 받은 연구가 그렇지 않은 연구에 비해 더 큰 효과 크기를 나타내었지만, 다른 5개 연구에서는 효과 크기 측면에서 별 차이가 없었다. (의료기기 연구 120 건이 포함된) 단 2건의 연구에서만, 의료기기에 대한 자료를 별도로 보고하였다. 약물과 의료기기 연구 간에는 기업 후원과 연구 결론의 상관성 측면에서 차이가 없었다 (상호작용 검정, P=0.23). 기업 후원을 받은 연구와 그렇지 않은 연구를 비교했을 때 무작위 순서 생성, 배정 순서 숨김, 추적관찰 측면에서 비뚤림 위험은 별 차이가 없었다. 그러나, 기업 후원을 받은 연구는 그렇지 않은 연구보다 더 자주 눈가림과 연관된 바이어스 위험이 낮았다 (RR 1.32, 95% CI 1.05 to 1.65) 연구 결과와 결론 간 일치도는 기업 후원을 받은 연구에서 그렇지 않은 연구보다 더 낮았다 (RR 0.84, 95% CI 0.70 to 1.01).

연구진 결론

제조회사가 후원한 약물 및 의료기기 연구는 그렇지 않은 연구보다 시험제품에 유리한 방향의 결과와 결론을 더 잘 이끌어낸다. 우리의 분석은 기존의 표준 '바이어스 위험' 평가로 설명할 수 없는 기업 후원 바이어스가 존재함을 제기한다.

역주

이 리뷰는 김건형 님 (부산대학교 한의학전문대학원) 이 번역하였습니다. 번역 내용과 관련한 문의점은 김건형 (pdchrist@gmail.com) 님에게 연락주세요.

Plain language summary

Industry sponsorship and research outcome

Results from clinical studies on drugs and medical devices affect how doctors practice medicine and thereby the treatments offered to patients. However, clinical research is increasingly sponsored by companies that make these products, either because the companies directly perform the studies, or fully or partially fund them. Previous research has found that pharmaceutical industry sponsored studies tend to favor the sponsors’ drugs much more than studies with any other sources of sponsorship. This suggests that industry sponsored studies are biased in favor of the sponsor’s products.

This review is an update of a previous review on this topic that looked only at drug studies. It uses more rigorous methodology and also investigates sponsorship of medical device studies. The primary aim of the review was to find out whether the published results and overall conclusions of industry sponsored drug and device studies were more likely to favor the sponsors’ products, compared with studies with other sources of sponsorship. The secondary aim was to find out whether such industry sponsored studies used methods that increase the risk of bias, again compared with studies with other sources of sponsorship. We did a comprehensive search of all relevant papers published before September 2010 and included 48 papers in our review.

Industry sponsored drug and device studies more often had favorable efficacy results, (risk ratio (RR): 1.32, 95% confidence interval (CI): 1.21 to 1.44), harms results (RR: 1.87, 95% CI: 1.54 to 2.27) and overall conclusions (RR: 1.31, 95% CI: 1.20 to 1.44), compared with non-industry sponsored drug and device studies. We did not find a difference between industry and non-industry sponsored studies with respect to standard factors that may increase the risk of bias, except for blinding: industry sponsored studies reported satisfactory blinding more often than non-industry sponsored studies. We did not find a difference between drug and device studies on the association between sponsorship and conclusions. In industry sponsored studies, there was less agreement between the results and the conclusions than in non-industry sponsored studies, RR: 0.84 (95% CI: 0.70 to 1.01). Our analysis suggests that industry sponsored drug and device studies are more often favorable to the sponsor’s products than non-industry sponsored drug and device studies due to biases that cannot be explained by standard 'Risk of bias' assessment tools.

Laienverständliche Zusammenfassung

Industriesponsoring und Forschungsergebnisse

Ergebnisse aus klinischen Studien zu Medikamenten und Medizinprodukten beeinflussen, wie Ärzte ihren Beruf ausüben, und dadurch auch die Behandlungen, welche Patienten angeboten werden. Die klinische Forschung wird jedoch zunehmend von Unternehmen gesponsert, welche diese Produkte herstellen, entweder weil diese Unternehmen solche Studien selbst durchführen oder sie voll oder teilweise finanzieren. Untersuchungen haben gezeigt, dass in Studien, die von der pharmazeutischen Industrie gesponsert werden, die Medikamente des Sponsors deutlich besser dargestellt werden als in Studien mit anderen Finanzierungsquellen. Dies deutet darauf hin, dass in industriegesponserten Studien ein systematischer Fehler (Bias) zugunsten der Sponsor-Produkte besteht.

Dieser Review ist eine Aktualisierung eines früheren Reviews zu diesem Thema, welcher nur Medikamentenstudien untersuchte. Im vorliegenden Review kommt zum einen eine striktere Methodik zur Anwendung, zum anderen wird auch die Finanzierung von Studien zu Medizinprodukten untersucht. Das Hauptziel bestand darin herauszufinden, ob die publizierten Ergebnisse und Schlussfolgerungen von industriegesponserten Studien zu Medikamenten und Medizinprodukten eher die Produkte des Sponsors begünstigten als Studien mit anderen Finanzierungsquellen. Darüber hinaus sollte herausgefunden werden, ob in solchen industriegesponserten Studien Methoden eingesetzt wurden, welche das Bias-Risiko erhöhen. Auch dazu wurden die industriegesponserten Studien mit Studien verglichen, die aus anderen Quellen finanziert wurden. Wir durchsuchten alle relevanten Arbeiten, welche vor September 2010 publiziert wurden, und konnten 48 dieser Arbeiten in diesem Review einschließen.

Industriegesponserte Studien zu Medikamenten und Medizinprodukten zeigten häufiger günstige Ergebnisse für Wirksamkeit (relatives Risiko (RR): 1,32, 95 % Konfidenzintervall (KI): 1,21 bis 1,44), Nebenwirkungen (RR: 1,87, 95 % KI: 1,54 bis 2,27) und Schlussfolgerungen (RR: 1,31, 95% KI: 1,20 bis 1,44), als nicht industriegesponserten Studien. Wir konnten im Hinblick auf bekannte Faktoren, welche das Bias-Risiko erhöhen, keinen Unterschied zwischen industriegesponserten und nicht industriegesponserten Studien finden, außer bei der Verblindung: Industriegesponserte Studien berichteten häufiger von ausreichender Verblindung als nicht industriegesponserte Studien. Wir konnten keinen Unterschied zwischen Studien zu Medikamenten und Medizinprodukten zum Zusammenhang zwischen Sponsoring und Schlussfolgerungen finden. In industriegesponserten Studien bestand weniger Übereinstimmung zwischen den Ergebnissen und den Schlussfolgerungen als in nicht industriegesponserten Studien [RR: 0,84 (95 % KI: 0,70 bis 1,01)]. Unsere Analyse deutet darauf hin, dass aufgrund von systematischen Fehlern, welche nicht durch gebräuchliche Instrumente zur Bestimmung des Bias-Risikos erklärt werden können, industriegesponserten Studien zu Medikamenten und Medizinprodukten häufiger Produkte des Sponsors begünstigen als nicht industriegesponserte Studien.

Anmerkungen zur Übersetzung

Koordination durch Cochrane Schweiz.

쉬운 말 요약

기업 후원이 연구결과에 미치는 영향

약물 및 의료 기기에 대한 임상연구 결과는 의사의 진료 방식에 영향을 주며, 이러한 영향은 환자가 받는 치료에도 반영된다. 한편 의약품 및 의료기기를 생산하는 기업이 지원하는 임상연구가 점점 더 많아지고 있는데, 회사가 직접 연구를 수행하거나 연구비를 전액 또는 부분적으로 지원하기 때문이다. 선행 연구에서는, 제약 기업이 후원한 연구가 다른 후원처를 가진 연구에 비해 그 기업의 약물에 유리한 결과를 내놓는 경향이 훨씬 더 크다는 것을 확인하였다. 이는 기업 후원을 받은 임상연구가 그 기업의 제품에 유리한 방향의 바이어스를 지니고 있음을 제기한다.

이 리뷰는 약물 임상연구의 분석에 국한되었던 기존 동일한 주제의 리뷰를 업데이트하였다. 우리는 보다 엄밀한 방법론을 적용하여 의약품과 의료 기기 임상연구에 대한 기업 후원을 조사하였다. 이 리뷰의 주요 목적은 기업 후원 약물 및 의료기기 연구에서 출판된 결과와 전반적 결론이, 기업이 아닌 다른 후원처를 가진 임상연구들에 비해 그 기업의 제품에 유리한 경향을 더 많이 띠는지 조사하는 것이다. 두 번째 목적은, 기업 후원 연구가 기업이 아닌 다른 후원처를 가진 연구에 비해 바이어스 위험을 증가시키는 연구방법을 사용했는지 조사하는 것이다. 우리는 2010년 9월 이전에 출간된 모든 관련 연구를 찾고자 포괄적 검색을 수행하였으며, 48개의 연구를 리뷰에 포함시켰다.

기업 후원 약물 및 의료기기 임상연구는, 기업이 아닌 후원처를 지닌 약물 및 의료기기 임상연구에 비해 효능 (RR 1.32, 95% CI 1.21 to 1.44), 위해 (RR 1.87, 95% CI 1.54 to 2.27), 전반적 결론 (RR 1.31, 95% CI 1.20 to 1.44) 측면에서 시험 약물 및 기기에 유리한 결과를 더 자주 나타내었다. 기업 후원 임상연구와 그렇지 않은 연구 간에는, 바이어스 위험 증가와 관련된 표준 요소 측면의 차이를 발견하지 못했다. 다만, 눈가림은 예외였다: 기업 후원 연구는 그렇지 않은 연구에 비해 눈가림을 더 만족스러운 수준으로 보고하였다. 기업 후원과 연구의 결론 사이 연관성은 약물 연구냐 의료기기 연구냐에 따라 달라지지 않았다. 기업 후원 연구에서는 그렇지 않은 연구에 비해, 연구 결과와 결론 간 일치도가 더 작았다 (RR 0.84, 95% CI 0.70 to 1.01). 이 리뷰는 기업 후원 약물 및 의료기기 연구가 그렇지 않은 약물 및 의료기기 연구에 비해 보다 자주 해당 기업의 제품에 유리한 성향을 띠며, 이는 표준 '바이어스 위험' 평가 도구로 설명할 수 없는 바이어스 때문임을 제기한다.

역주

이 리뷰는 김건형 님 (부산대학교 한의학전문대학원) 이 번역하였습니다. 번역 내용과 관련한 문의점은 김건형 (pdchrist@gmail.com) 님에게 연락주세요.

Laički sažetak

Istraživanja koja financira farmaceutska industrija češće imaju rezultate koji idu u prilog sponzorovom proizvodu

Rezultati kliničkih istraživanja lijekova i medicinskih proizvoda utječu na način kako liječnici prakticiraju medicinu i na terapije koje propisuju pacijentima. Međutim, klinička istraživanja sve češće sponzoriraju tvrtke koje proizvode te lijekove i proizvode, ili zbog toga što tvrtke same provode te studije ili zato jer ih potpuno ili djelomično financiraju. Prethodna su istraživanja pokazala da studije koje financira farmaceutska industrija češće imaju rezultate koji idu u prilog sponzorovom lijeku nego studije koje imaju druge izvore financiranje. To ukazuje na to da su studije koje sponzorira financijska industrija pristrane prema proizvodu sponzora.

Ovaj sustavni pregled je obnovljena verzija ranije objavljenog sustavnog pregleda na ovu temu koji je ispitao samo studije o lijekovima. Nova verzija koristi bolju metodologiju te također ispituje sponzorstvo studija koje su ispitale medicinske proizvode. Glavni cilj ovog sustavnog pregleda bio je istražiti da li objavljeni rezultati i cjelokupni zaključci studija o lijekovima i medicinskim proizvodima koje financira industrija imaju veću vjerojatnost davanja pozitivnih rezultata za sponzorove lijekove i proizvode u usporedbi sa studijama koje imaju druge izvore financiranja. Drugi je cilj bio istražiti da li studije koje sponzorira industrija koriste metode koje povećavaju rizik od pristranosti u usporedbi sa studijama koje se financiraju iz drugih izvora. Temeljito je pretražena literatura kako bi se pronašli svi relevantni radovi na tu temu, a koji su objavljeni do rujna 2010., i u ovaj sustavni pregled je uključeno 48 studija.

Studije o lijekovima i medicinskim proizvodima koje je financirala industrija češće su imali povoljne rezultate o djelotvornosti, štetnosti i povoljne ukupne zaključke u usporedbi sa studijama o lijekovima i medicinskim proizvodima koje nije financirala industrija. Nije pronađena razlika između studija koje jest ili nije financirala industrija u smislu korištenja metoda koje povećavaju rizik od pristranosti, osim u smislu zasljepljenja (zasljepljenje znači da ispitanici i osobe koje ih prate ne znaju koju terapiju primaju); studije koje je sponzorirala industrija češće su opisivale propisno zasljepljivanje ispitanika i medicinskog osoblja nego studije koje nije financirala industrija. Nisu utvrđene razlike između studija o lijekovima i studija o medicinskim proizvodima u smislu povezanosti između sponzorstva i zaključaka. U studijama koje je financirala industrija bilo je manje slaganja između rezultata i zaključaka nego u studijama koje nije financirala industrija. Ova analiza ukazuje da studije o lijekovima i medicinskim proizvodima koje financira industrija češće imaju zaključke koji idu u prilog sponzorovom proizvodu nego studije lijekova i medicinskih proizvoda koje imaju druge izvore financiranja, i to zbog pristranosti koja se ne može objasniti standardnom procjenom "rizika od pristranosti".

Bilješke prijevoda

Hrvatski Cochrane
Prevela: Livia Puljak
Ovaj sažetak preveden je u okviru volonterskog projekta prevođenja Cochrane sažetaka. Uključite se u projekt i pomozite nam u prevođenju brojnih preostalih Cochrane sažetaka koji su još uvijek dostupni samo na engleskom jeziku. Kontakt: cochrane_croatia@mefst.hr

Background

Description of the problem or issue

Clinical research sponsored by the pharmaceutical industry affects how doctors practice medicine (PhRMA 2008; Wyatt 1991). An increasing number of clinical trials at all stages in a product's life cycle are funded by the pharmaceutical industry, and the industry now spends more on medical research than do the National Institutes of Health in the United States (Dorsey 2010). Results and conclusions that are unfavorable to the sponsor (i.e. studies that find an expensive drug similarly or less effective or more harmful than drugs used to treat the same condition) can pose considerable financial risks to companies.

Several systematic reviews have documented that pharmaceutical industry sponsorship of drug studies is associated with findings that are favorable to the sponsor’s product (Bekelman 2003; Lexchin 2003; Schott 2010; Sismondo 2008a). There are several ways that industry can sponsor a study, including single-source sponsorship, shared sponsorship, and provision of free drugs or devices only. There are also several potential ways that industry sponsors can influence the outcome of a study, including the framing of the question, the design of the study, the conduct of the study, how data are analyzed, selective reporting of favorable results, and spin in reporting conclusions (Bero 1996; Lexchin 2012; Sismondo 2008b). Although some journals now require that the role of the sponsor in the design, conduct and publication of the study be described, this practice is not widespread (Tuech 2005).

Why it is important to do this review

This systematic review is the update of an original systematic review by two of the authors (Lexchin 2003), which investigated whether sponsorship by industry is associated with the publication of outcomes favorable to the sponsor. That review is now out of date and included pharmacoeconomic papers. We therefore updated it. Recent developments, such as the adoption of trial registration could lessen the bias associated with industry sponsorship, as publication bias can be more readily detected (DeAngelis 2004). On the other hand, the release of internal industry documents as a result of settlement agreements resulting from litigation against drug companies has revealed examples of industry manipulation of the conduct and publication of studies (Fugh-Berman 2010; Ross 2008; Steinman 2006; Vedula 2009). In addition, the scope of the review is now expanded to include device studies, as they are subject to the same biases as drug studies and are also often sponsored by companies with a financial interest in the outcome.  

Objectives

The objectives were to investigate whether:

  • sponsorship of drug and device studies by the pharmaceutical and device industries is associated with outcomes, including conclusions, that are favorable to the sponsor;

  • drug and device studies sponsored by the pharmaceutical and device industries differ in their risk of bias compared with studies with other sources of sponsorship.

Methods

Criteria for considering studies for this review

Types of studies

This review includes reports of studies that investigate samples of primary research studies. To avoid confusion we will use the terms 'studies' for the primary research studies and 'papers' for the reports of studies of primary research studies. We will use the term trials to describe studies of a randomized clinical trial design.

We included papers of cross-sectional studies, cohort studies, systematic reviews or meta-analyses that quantitatively compared primary research of drug or medical device studies sponsored by the pharmaceutical or device industry with studies that had other sources of sponsorship. Drugs were defined as medications that require approval by a regulatory authority as a prescription drug, recognizing that these approval standards vary worldwide. Devices were defined based on the Food and Drug Administration (FDA) definition as instruments intended for use in the diagnosis, treatment or prevention of disease.

We excluded papers without quantitative data. We excluded papers of the effects of sponsorship by non-pharmaceutical or non-device (e.g. tobacco, food or chemical) industries, and papers that evaluated the effectiveness of herbal supplements or medical procedures. Papers of mixed interventions (e.g. pharmaceuticals and educational interventions) were included if drug or device data were reported separately or could be obtained from the authors.

We excluded papers that quantitatively compared the association of sponsorship and results of syntheses of research studies (i.e. systematic reviews or meta-analyses) or pharmacoeconomic studies of drugs or devices. We also excluded analyses of pharmacokinetic studies.

Only papers published in full were included; we excluded letters to the editor and published conference presentations. We had no language restrictions.

Types of data

Drug and device papers including human research studies comparing drug to placebo, device to sham, drug to drug, drug to device, device to device, or mixed comparisons where the effectiveness, efficacy or harms of the drug or device were evaluated.

Types of methods

We defined sponsorship as funding or provision of free drug or devices. Drug or device studies with pharmaceutical or device industry funding versus those with other or undisclosed funding were included. We extracted the definition of industry funding verbatim from the included papers (see Data extraction and management) and reported this in the 'Characteristics of included studies' table. For analysis, we grouped the definitions into a variety of categories, including 100% pharmaceutical or device company funding, 100% non-profit funding, mixed funding (e.g. non-profit and industry collaboration), free provision of drug or device only, and undisclosed funding.

We included papers that compared industry sponsored studies with non-industry sponsored studies and also papers that compared studies of products by competing manufacturers (i.e. studies sponsored by the manufacturer of the test treatment with studies sponsored by the manufacturer of the control treatment); we analyzed the two types of papers separately.

Types of outcome measures

Primary outcomes

We included two primary outcomes:

  1. Whether the results were favorable to the sponsor.

  2. Whether the conclusions were favorable to the sponsor. 

We used the definition of favorable results as described in the methods of the included papers. For efficacy results, most papers considered favorable results to be those that were statistically significant (e.g. P < 0.05 or 95% confidence interval excluding the possibility of no difference) in favor of the sponsor's product. Based on the previous review (Lexchin 2003), which found very few studies that reported results unfavorable to the sponsor, unfavorable results were combined with studies that reported results that were neutral or not statistically significant. For harms results, most papers regarded favorable results to be those where harms were not statistically significant (e.g. P > 0.05 or 95% confidence interval including the possibility of no difference) or results that had a statistically significant higher number of harms in the comparator group.

Conclusions in which the sponsor’s product was preferred over the control treatment were considered favorable to the sponsor. For conclusions we did not distinguish between efficacy and harms, as conclusions are often overall qualitative judgements based on a benefit to harm balance.

Secondary outcomes

We included three secondary outcomes.

  1. The size of the effect estimate in industry sponsored studies versus those with other sources of sponsorship.

  2. The risk of bias in industry sponsored studies versus those with other sources of sponsorship.

  3. The concordance between study results and conclusions, i.e. whether the conclusions agreed with the study results, in industry sponsored studies versus those with other sources of sponsorship. 

We included papers that reported at least one of these secondary outcomes, even if it reported neither of the primary outcomes.

Search methods for identification of studies

Electronic searches

We searched Ovid MEDLINE (R) In-Process and other non-indexed citations and Ovid MEDLINE (R) (1948 to September 2010), Ovid EMBASE (1980 to September 2010) and the Cochrane Methodology Register (Issue 4, 2010) (Wiley InterScience Online). We searched the Web of Science (August 2011) for papers that cited any of the papers included in our review.

Search strategy

We used the strategy shown in Appendix 1 for Ovid MEDLINE and adapted it for the other databases.

Searching other resources

Other sources of data included author files, searches of reference lists of included papers and previous systematic reviews.

Data collection and analysis

Selection of studies

Two assessors  (AL and OAB) screened the titles and abstracts, when available, of all retrieved records for obvious exclusions, and assessed the remaining papers based on full text. Potentially eligible papers were sent to the other assessors for final validation of the inclusion criteria. Any disagreements were resolved by consensus and reasons for exclusions of potentially eligible papers are described in the 'Characteristics of excluded studies' table. There was no need for translation of non-English papers.

Data extraction and management

Two assessors (AL and SS) independently extracted data from included papers; differences in data extraction were resolved by consensus. 

We extracted data on the following.

  • Year published.

  • Country of corresponding author.

  • Study objective.

  • Study design used in the paper (cohort, cross-sectional, systematic review or meta-analysis, other).

  • Study domain - descriptive (e.g. oncology drug trials).

  • Study domain - category (drug/device class, specific disease, medical specialty/type of diseases, mixed).

  • Type of studies (drug, device, drug and device, mixed).

  • Type of comparisons (drug versus drug, drug versus placebo, device versus device, device versus sham, device versus drug, mixed, other).

  • Sample strategy used to locate research studies (electronic search only, electronic plus other, sampling of journals, sampling by venue (e.g. conference abstracts)).

  • Whether there were language restrictions on the search.

  • Number of studies included in the sample.

  • Time period covered by studies in the paper.

  • Sponsorship categories coded in the paper. Categories were:

    • 100% pharmaceutical/device company funded;

    • 100% non-profit funded;

    • mixed funding - e.g. non-profit and industry collaboration;

    • provision of drug or device only; and

    • undisclosed funding.

  • Sponsorship categories used in analysis in the paper (e.g. 100% industry funded grouped with mixed funding for industry category).

  • Data on association between author conflicts of interest and outcomes.

  • Description of role of the sponsor (if any). For example, definition of the sponsor’s role in the design, implementation or reporting in the sample of studies.

  • Criteria used to assess risk of bias of the studies included in the paper.

  • Primary purpose of the study.

  • Whether the paper commented on appropriateness of comparators.

  • Data on sponsorship and results.

  • Data on sponsorship and conclusions.

  • Data on sponsorship and effect size.

  • Data on sponsorship and risk of bias.

  • Data on sponsorship and concordance between study results and conclusions.

  • Additional relevant data.

Assessment of risk of bias in included studies

Since there are no validated criteria for assessing risk of bias in these types of papers, we developed our own criteria. We reviewed papers for high, low or unclear risk of bias for each of four criteria. If a criterion was met it was regarded as having low risk of bias, and high risk of bias otherwise. If we could not determine whether a criterion was met, we coded it as unclear. We used the following criteria:

  • whether explicit and well defined criteria that could be replicated by others were used to select studies for inclusion/exclusion;

  • whether there was an adequate study inclusion method, with two or more assessors selecting studies;

  • whether the search for studies was comprehensive; and

  • whether methodological differences and other characteristics that could introduce bias were controlled for or explored.

Measures of the effect of the methods

We performed a meta-analysis of the papers that reported the association of sponsorship with favorable study outcomes in cases where a pooled risk ratio (RR) and its 95% confidence interval could be computed.

The definition of a favorable outcome varied among papers. In some papers it was stated that favorable outcomes were outcomes favorable to the sponsor's product and in others favorable to the test treatment. This difference in terminology did not matter when the comparison was between active treatment and placebo, since the sponsor was related to the active treatment and not placebo. For head-to-head comparisons, however, the sponsor could be either the manufacturer of the test treatment or the control treatment. In these cases, when data were available, we recoded outcomes as to whether they were favorable to the sponsor's product.

We separately analyzed papers of industry sponsored head-to-head studies, comparing studies sponsored by the manufacturer of the test treatment with studies sponsored by the manufacturer of the comparator treatment. This was done by assigning the newest treatment (most recent FDA approval date) as the 'test' treatment and the older treatment as the 'comparator' treatment using similar methods as described by Bero et al. (Bero 2007) and comparing the number of studies favorable to the test treatment in the two groups (i.e. sponsor produces test treatment or sponsor produces comparator treatment).

At the time many of the papers were conducted, the approach was to assess the methodological quality of studies as opposed to an assessment of the risk of bias of studies. We therefore recoded the data on methodological quality into 'Risk of bias' categories. So, for example, a trial with adequate concealment of allocation was coded as low risk of bias and a trial with inadequate concealment of allocation as high risk of bias. Some papers assessed risk of bias by summarizing the information of individual domains in an overall methodological quality score (i.e. a scale approach). There are substantial methodological problems related to quality scales (Jüni 1999) and their use is not recommended. We therefore did not combine the results obtained with these scales, but report the results descriptively.

Dealing with missing data

We contacted authors of the original papers in an attempt to obtain missing data. If papers included studies reporting conflicts of interest, but not the source of funding, we contacted the authors in order to obtain separate data for funding. In total we contacted authors of 36 papers and received additional data for 18 of these papers.

Assessment of heterogeneity

We assessed heterogeneity using I2. In our initial protocol we intended to use a random-effects model when P was < 0.10 for the Chi2 test. However, since this is not in line with current recommendations in the Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (Higgins 2011a), we instead used a random-effects model when statistical heterogeneity was substantial, defined as an I2 > 50%.

Data synthesis

We used Review Manager (RevMan 2011) to analyze data. For dichotomous data we used the Mantel-Haenszel fixed-effect model to create a pooled RR. However, when substantial heterogeneity was observed, we used an inverse variance DerSimonian-Laird random-effects model.

Subgroup analysis and investigation of heterogeneity

We considered the following factors as potential explanations for heterogeneity and investigated them in separate subgroup analyses.    

  1. We hypothesized that the association of industry sponsorship and favorable outcomes may be larger in high risk of bias papers. We assessed overall risk of bias of the included papers using the criteria described in 'Assessment of risk of bias in included studies'. We regarded papers with adequate study inclusion, a comprehensive search and controlling for bias as having a low risk of bias; others as having a high risk. We compared low risk of bias papers with high risk of bias papers in a subgroup analysis.

  2. We compared papers of drug studies with device studies, as the mechanisms of influencing study outcomes may differ between the industries. For example, drug trials are more regulated than device trials, which could have an influence on biases in the design, conduct and reporting of the trials. We compared this in a subgroup analysis.   

  3. As the study domain might contribute to heterogeneity, we compared papers on specific treatments or diseases with papers of mixed domains in another subgroup analysis.

Sensitivity analysis

We undertook the following sensitivity analyses to test the robustness of our findings.

  1. The primary analyses compared the number of favorable results and conclusions in papers with industry sponsorship to those with other sources of sponsorship; 'industry sponsorship' included 100% pharmaceutical/device company funding, mixed funding and provision of drug or device only. 'Non-industry sponsorship' included 100% government funding, 100% non-profit funding and undisclosed funding. In a sensitivity analysis, we excluded those studies with mixed funding sources and those with funding consisting solely of free product from the 'industry sponsorship' category, and excluded studies with undisclosed funding from the category of 'non-industry sponsorship', to determine if these had an impact on the initial analysis. As noted under 'Data extraction and management' we were reliant on how the studies in our review defined 'funding'.

  2. Originally we had intended a sensitivity analysis restricted to papers with a low risk of bias using estimates adjusted for confounders (e.g. adjusted for sample size and concealment of allocation using logistic regression). However, because few papers with low risk of bias reported adjusted estimates in a way that we could use in our analysis, we decided to base our analysis on both low and high risk of bias papers reporting adjusted estimates. We used the generic inverse variance method to pool adjusted odds ratios in a fixed-effect model.

  3. Due to the variability in study characteristics and methodology between papers, a random-effects model may be preferred, even if no statistical heterogeneity is observed. We therefore also undertook a sensitivity analysis where all analyses were based on a random-effects model.

Results

Description of studies

See:Characteristics of included studies; Characteristics of excluded studies.

Results of the search

See: Figure 1.

Figure 1.

Study flow diagram.

After removal of duplicates, 2579 references were identified. From reading titles and abstracts, 2432 were eliminated as being not relevant to the review. Full-text papers were obtained for 147 references. From these 147 papers, 73 papers were excluded and 74 were retained for assessment by all assessors. Of these 74 papers, 30 were excluded (see Characteristics of excluded studies) and 44 included (see Characteristics of included studies). One additional paper (Chard 2000) was included as a result of searching reference lists of previous systematic reviews and three from searching Web of Science for papers citing any of the included papers (Jones 2010; Lubowitz 2007; Pengel 2009).

Included studies

See: Characteristics of included studies.

The 48 papers were published between 1986 and 2010. Forty-six papers included mainly published studies, one included studies presented at a conference, and one included studies submitted to a medical journal. Thirty-seven papers included only drug studies, one only device studies, one drug and device studies and nine included different types of interventions (e.g. drugs, devices, behavioral interventions). Nineteen papers included studies related to specific drug classes, 13 related to specific medical specialties or types of diseases (e.g. endocrinology), six related to a specific disease, one related to a specific type of device, eight included all types of research studies and one did not state the domain. Various aspects of medicine were covered, but 10 (21%) papers were restricted to psychiatric diseases or drugs. Thirty-five papers included only clinical trials, two only observational studies, and 11 both clinical trials and observational studies. Eight papers included only drug versus drug comparisons, three only drug versus placebo, 34 mixed comparisons (e.g. drug versus drug, drug versus placebo) and three did not describe the kind of comparisons. The median number of included studies per paper was 137 (range: nine to 930). Of the 48 papers, 16 reported data on both favorable outcomes and risk of bias, 28 on favorable outcomes only and four on risk of bias only.

Risk of bias in included studies

See: Figure 2; Figure 3.

Figure 2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Figure 3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Forty-four papers had low risk of bias for the selection criteria for inclusion of studies, three were unclear and one had high risk. Ten papers had low risk of bias for the study inclusion process, 31 were unclear and seven had high risk. Forty-six papers had low risk of bias from the search and two had high risk. Twenty-three papers had low risk of bias due to lack of control for bias in the studies, three were unclear and 22 had high risk. Nine papers were regarded as having an overall low risk of bias and 39 as a high risk of bias according to our criteria.

Effect of methods

Favorable results: industry sponsored versus non-industry sponsored studies

Fifteen papers, including 1746  studies (all drug studies), reported on sponsorship and efficacy results, and 14 could be combined in a pooled analysis. An analysis based on these 14 papers, including 1588 studies, found that industry sponsored studies more often had favorable efficacy results (e.g. those with significant P values) compared with non-industry sponsored studies, risk ratio (RR): 1.32 (95% confidence interval (CI): 1.21 to 1.44), I2: 42% (Analysis 1.1). The paper that could not be included in the pooled analysis (Bhandari 2004), which had included 158 drug studies in general medicine, found similar results, odds ratio (OR): 1.6 (95% CI: 1.1 to 2.8).

Three papers, including 561 studies, found that industry sponsored studies more often had favorable harms results compared with non-industry sponsored studies, RR: 1.87 (95% CI: 1.54 to 2.27). No heterogeneity was observed (Analysis 1.2). The analysis was driven by one study (Nieto 2007) that contributed 97% of the weight in the analysis.

Favorable results: industry sponsorship by test treatment company versus industry sponsorship by comparator treatment company

Three papers, including 151 trials (all drug trials), compared efficacy results of trials sponsored by the manufacturer of the test treatment with trials sponsored by the manufacturer of the comparator treatment, and two could be combined in a pooled analysis. An analysis based on these two papers (Bero 2007; Rattinger 2009), which included 131 industry sponsored trials of statins and thiazolidinediones, found that trials were much more likely to favor the test treatment when they were sponsored by the manufacturer of the test treatment than when they were sponsored by the manufacturer of the comparator treatment, RR: 4.64 (95% CI: 2.08 to 10.32), I2: 50% (Analysis 2.1). The paper that could not be included in the pooled analysis, which had included 20 selective serotonin reuptake inhibitor head-to-head trials, found that two trials favored the sponsor's drug, 18 had similar efficacy and none favored the comparator drug (Gartlehner 2010).

Favorable conclusions: industry sponsored versus non-industry sponsored studies

Twenty-four papers, including 4616 studies (4403 drug studies and 213 device studies), reported on sponsorship and conclusions, and 21 could be combined in a pooled analysis. An analysis based on these 21 papers, including 3941 studies (3821 drug studies and 120 device studies), found that industry sponsored studies more often had favorable conclusions than non-industry sponsored studies, RR: 1.31 (95% CI: 1.20 to 1.44), I2: 83% (Analysis 3.1). Three papers could not be included in the pooled analysis. Of these, one paper of 301 psychiatric drug studies (Kelly 2006) found that industry sponsored studies more often had favorable conclusions than non-industry sponsored studies (P < 0.001) and similar findings were reported in a paper of 59 trials of antipsychotics (P = 0.02) (Montgomery 2004). A paper of 315 gastroenterology trials (222 drug trials and 93 device trials) did not find a difference in conclusions between industry sponsored trials and non-industry sponsored trials (industry: 86% favorable, non-industry: 83% favorable; P = 0.57) (Brown 2006).

Favorable conclusions: industry sponsorship by test treatment company versus sponsorship by comparator treatment company

Five papers, including 348 drug trials, compared conclusions of studies sponsored by the manufacturer of the test treatment with studies sponsored by the manufacturer of the comparator treatment, and three could be combined in a pooled analysis. An analysis based on these three papers (Bero 2007; Heres 2006; Rattinger 2009) including 154 industry sponsored trials of statins, antipsychotics and thiazolidinediones, found that trials were much more likely to favor the test treatment when they were sponsored by the manufacturer of the test treatment than when they were sponsored by the manufacturer of the control treatment, RR: 5.90 (95% CI: 2.79 to 12.49). No heterogeneity was observed (Analysis 4.1). A paper of 138 psychiatric drug studies (Kelly 2006) had similar findings, RR 2.80 (95% CI: 2.02 to 3.88), and a paper of 56 non-steroidal anti-inflammatory drug (NSAID) trials (Rochon 1994) found that 16 trials favored the sponsor's drug, 40 concluded that the drugs had similar effect and none favored the comparator drug.

Effect size: industry sponsored versus non-industry sponsored studies

Ten papers, including 906 studies (865 drug studies and 41 device studies), reported on sponsorship and effect size, but could not be pooled due to differences in their reporting of data. The results were heterogeneous.

Five papers, including 798 drug studies, did not find a difference in effect size between industry sponsored studies and non-industry sponsored studies. One paper including 370 drug trials (Als-Nielsen 2003) found mean z-scores of -1.48 (95% CI: -1.19 to -1.77) in industry sponsored trials, -1.77 (95% CI: -1.35 to -2.28) in trials with mixed sponsorship and -1.20 (95% CI: -0.59 to -1.81) in non-industry sponsored trials, which were not statistically significantly different. Similarly, a paper of 176 trials of drugs for acute pain and migraine (Barden 2006) did not find a difference in number of patients with pain relief between industry sponsored trials and non-industry sponsored trials. A paper of 124 trials comparing second-generation antipsychotics with first-generation antipsychotics (Davis 2008) did not find a difference in effect size between industry sponsored trials and non-industry sponsored trials (P = 0.57). A paper of 105 trials comparing selective serotonin reuptake inhibitors with alternative antidepressants (Freemantle 2000) also did not find a difference in effect size between industry sponsored trials and non-industry sponsored trials. A paper of 23 studies of chondrocyte implantation (Lubowitz 2007) did not find a difference in effect size between industry sponsored studies and non-industry sponsored studies for various outcomes.

In contrast, four papers found higher effects in industry sponsored studies. A paper including nine trials comparing clozapine with conventional antipsychotics (Moncrieff 2003) found that the treatment effect was higher in industry sponsored trials than in non-industry sponsored trials, standardized mean difference (SMD): -0.83 (95% CI: -1.06 to -0.61) versus SMD: -0.21 (95% CI: -0.34 to -0.07) (P < 0.001). Similarly, a paper of 41 dental implant trials (Popelut 2010) found that the failure rates were lower in industry sponsored trials compared with non-industry sponsored trials, OR: 0.21 (95% CI: 0.12 to 0.38). A paper including 15 trials of glucosamine (Vlad 2007) found that the effect size was higher in industry sponsored trials than in non-industry sponsored trials, SMD: 0.47 (95% CI: 0.24 to 0.70) versus SMD: 0.05 (95% CI: -0.32 to 0.41) (P = 0.05). One paper including 34 nicotine replacement drug trials (Etter 2007) found higher effects in industry sponsored trials compared with non-industry sponsored trials, OR: 1.90 versus OR 1.61 (P = 0.06).

Only one paper assessed effect size of harms (Kemmeren 2001). It included nine observational studies that compared third generation with second-generation oral contraceptives and found that the risk of thrombosis was lower in industry sponsored studies compared with non-industry sponsored studies, OR: 1.3 (95% CI: 1.0 to 1.7) versus OR 2.3 (95% CI: 1.7 to 3.2).

Risk of bias: industry sponsored versus non-industry sponsored studies

Nine papers, including 1505 studies (1327 drug studies, 178 device studies), measured risk of bias using five different composite quality scales (Brown, Cho, Cochrane, Jadad or Sackett) and the results were heterogeneous. Four papers did not find a difference in risk of bias between industry sponsored and non-industry sponsored studies (Cho 1996; Jefferson 2009; Lynch 2007; Vlad 2007), whereas five papers found lower risk of bias (i.e. higher methodological quality scores) in industry sponsored studies (Brown 2006; Djulbegovic 2000; Montgomery 2004; Pengel 2009; Perlis 2005a).   

Three papers, including 487 drug trials, did not find a difference in low risk of bias from sequence generation in industry sponsored trials compared with non-industry sponsored trials, RR: 0.85 (95% CI: 0.52 to 1.41), I2: 86% (Analysis 5.1). Ten papers, including 1311 drug trials, did not find a difference in low risk of bias from concealment of allocation in industry sponsored trials compared with non-industry sponsored trials, RR: 1.09 (95% CI: 0.86 to 1.38), I2: 54% (Analysis 5.2). Nine papers, including 1216 drug trials, found that industry sponsored trials more often had low risk of bias from blinding compared with non-industry sponsored trials, RR: 1.32 (95% CI: 1.05 to 1.65), I2: 74% (Analysis 5.3). Two papers, including 118 drug trials, did not find a difference in low risk of bias from loss to follow-up in industry sponsored trials compared with non-industry sponsored trials, RR: 0.98 (95% CI: 0.84 to 1.16). No heterogeneity was observed (Analysis 5.4).

Concordance between study results and conclusions: industry sponsored versus non-industry sponsored studies

Five papers, including 667 drug studies, reported on concordance between study efficacy results (e.g. as judged by their P values) and conclusions. Industry sponsored studies were less concordant than non-industry sponsored studies, RR: 0.84 (95% CI: 0.70 to 1.01), I2: 67% (Analysis 6.1). One paper (Alasbali 2009), including 39 drug studies, found markedly higher lack of concordance in industry studies than the other four papers, and this was the reason for the high heterogeneity between papers.

One paper, of 211 corticosteroid studies with statistically significant harms results, found that industry sponsored studies more often concluded that the drug was safe than non-industry sponsored studies, RR: 3.68 (95% CI: 2.14 to 6.33) (Nieto 2007).

Subgroup analysis and investigation of heterogeneity

Because only three papers with efficacy results data had low risk of bias (Bero 2007; Bourgeois 2010; Etter 2007) and only four with conclusions data had low risk of bias (Als-Nielsen 2003; Bero 2007; Finucane 2004; Jefferson 2009) our comparison of low and high risk of bias papers was limited. Nonetheless, the association between industry sponsorship and favorable results was stronger in the low risk of bias group than in the high risk of bias group, RR: 1.53 (95% CI: 1.32 to 1.78) versus 1.19 (95% CI: 1.06 to 1.33) (test for subgroup differences P = 0.008) (Analysis 7.1). For conclusions, the differences between the groups went in the same direction, RR: 1.54 (95% CI: 1.24 to 1.91) versus 1.26 (95% CI: 1.14 to 1.39), (test for subgroup differences P = 0.10) (Analysis 7.2).

Similarly, as only two papers (Lynch 2007; Ridker 2006) had data on device studies, the comparison between drug and device studies was limited. We did not find a difference in the association between sponsorship and conclusions in drug studies compared with device studies (Analysis 7.3). Only two papers with results data (Bourgeois 2010; Clifford 2002) and four with conclusion data (Buchkowsky 2004; Cho 1996; Davidson 1986; Kjaergard 2002) were of mixed domain. We did not find a difference in the association between sponsorship and results or conclusion in studies limited to specific treatments or diseases compared with studies of mixed domains (Analysis 7.4; Analysis 7.5).

Sensitivity analysis

Our re-analyses of the outcomes using variations in definition of sponsorship categories gave similar results as our main analyses for results, conclusions, sequence generation, concealment of allocation and blinding (Analysis 8.1; Analysis 8.2; Analysis 8.3; Analysis 8.4; Analysis 8.5). Our analyses based on pooling adjusted odds ratios confirmed our findings that industry sponsored trials compared with non-industry sponsored trials more often had favorable results, OR: 3.86 (95% CI: 1.93 to 7.70) and favorable conclusions, OR: 4.15 (95% CI: 2.40 to 7.19). No heterogeneity was observed (Analysis 8.6; Analysis 8.7). Similarly, the change from a fixed-effect model to a random-effects model did not affect our analyses (Analysis 8.8; Analysis 8.9; Analysis 8.10; Analysis 8.11; Analysis 8.12).

Discussion

Summary of main results

We found that drug and device studies sponsored by the manufacturing company more often had favorable results (e.g. those with significant P values) and conclusions than those that were sponsored by other sources. The findings were consistent across a wide range of diseases and treatments. We did not find any differences in risk of bias of drug and device trials sponsored by industry compared with non-industry sponsored trials, except in relation to blinding, where industry sponsored trials seemed to have lower risk of bias. The evidence from device studies was limited, but the association between sponsorship and outcomes was similar to drug studies.

Reasons for observed heterogeneity

For the association between sponsorship and favorable results of drug and device studies the data had acceptable heterogeneity, but heterogeneity for conclusions was substantial with an I2 of 83%.

One reason for this was likely that the coding of favorable results was similar across the different papers, using statistical significance as the cut-off, but coding varied for conclusions. Some papers did not describe what they considered a favorable conclusion and others used scales, but for similar scales the cut-off varied between papers. For example, on the same six-point scale one paper used four as cut-off (Djulbegovic 2000) and another six as cut-off (Als-Nielsen 2003).

Also, the proportion of studies with favorable conclusions in the non-industry sponsored group might have contributed to the size of the association and thereby the heterogeneity. For example, while the Chard and Liss papers (Chard 2000; Liss 2006) had a similar proportion of favorable industry sponsored studies (both 98%), they reported very different proportions of favorable non-industry sponsored studies (32% and 97%) and this explains why the risk ratios reported in the two studies were not the same: 3.03 in Liss and 1.01 in Chard. Variations in study domain or definition of favorable conclusions might explain why the risk ratios reported in the two papers were not similar. For example, in the Chard paper, a conclusion was coded as favorable if the study authors supported the use of the treatment, even in the absence of a statistical significant result. Our subgroup analysis to test for differences in the association of sponsorship and results or conclusions between studies of mixed domains and studies related to specific treatments or diseases did not show different results, though this was a simplistic comparison.

Our data for the relationship between sponsorship and effect size showed mixed results, with most not finding a difference. All but one of these papers were restricted to specific treatments, which may explain the different findings. A recent study of systematic reviews of nine different drugs found that the influence of reporting biases on effect sizes varied considerably between drugs (Hart 2012). Furthermore, one paper found that even when adjusting for effect size, industry sponsored studies more often had favorable conclusions, compared with non-industry sponsored studies (Als-Nielsen 2003). Therefore, while the direction of the relationship between sponsorship and favorable outcomes was consistent, the size of the effect likely varies depending on domains.

Reasons for favorable outcomes in industry sponsored studies

The pharmaceutical and medical device industries have strong interests in scientific publications that present their products positively, as publications are the basis of regulatory, purchasing, and medical decisions. These interests can influence the design, conduct and publication of studies in ways that make the sponsor’s product appear better than the comparator product (Bero 1996).

Several possible factors can explain the relationship between industry sponsorship and favorable outcomes. It has been argued that since many industry sponsored studies are undertaken to fulfill regulatory requirements, industry sponsored studies could have a lower risk of bias than non-industry sponsored studies (Rosefsky 2003). Even if this were true, it would not explain the association of industry sponsorship and favorable results and conclusions. In addition, we did not find evidence for differences in risk of bias except in relation to blinding, where industry sponsored trials tended to have a lower risk of bias, even when restricted to head-to head trials (Bero 2007). The papers comparing blinding between trials with different sponsorship often used a description of double blinding as an indicator for low risk of bias. Double blinding is an inconsistent term and does not ensure that, for example, outcome assessors are blinded (Devereaux 2001). The more frequent use of double blinding may therefore be a reporting issue, with industry trials being better reported. This is further substantiated by the fact that nearly all the papers finding a higher methodological quality score in industry studies used the Jadad scale, a scale which has been criticized for having more focus on the quality of reporting than on methodological quality (Lundh 2008).   

On the other hand, evidence suggests that for non-industry trials, companies may prevent proper blinding by restricting access to placebo drugs (Christensen 2012) and therefore differences in adequate blinding may be real. In addition, double blinding can be used as a proxy for low risk of bias and trials without double blinding are on average more likely to have favorable results (Pildal 2007). The effect of this bias is in the opposite direction of our findings, as it would lead to industry sponsored studies having less favorable results and conclusions, and our findings can therefore, not be explained by differences in risk of bias between industry and non-industry sponsored studies.

Another possible explanation for our findings could be that industry studies have larger sample sizes, and would have a higher chance of achieving statistically significant results. Although industry trials seem in general to be of larger size (Als-Nielsen 2003; Booth 2008; Bourgeois 2010; Etter 2007; Perlis 2005a), when we restricted our analysis to studies controlling for sample size and other confounders, the relationship between industry sponsorship and favorable results or conclusions was still present.

Industry argues that the trials they sponsor are more likely to have favorable results because they fund research that has a high chance of achieving success (Palmer 2003). However, when independent investigators conduct non-industry sponsored trials, they in most cases test treatments that have been approved based on favorable industry trial results. Non-industry sponsored trials would therefore also be expected to achieve successful results, unless they are designed to answer different questions than industry sponsored trials. For example testing a new treatment against a well-established treatment instead of against placebo or against an outdated, inferior treatment.

Accordingly, it seems most plausible that industry achieves overly positive results through a variety of biasing choices in the design, conduct and reporting of their studies. For example, industry protocols might include inferior comparators that will increase the chance of their product’s success. Djulbegovic et al. (Djulbegovic 2003) have argued that industry sponsored studies violate equipoise by choosing inferior competing treatment alternatives. Previous studies have found that industry sponsored trials more often use placebo control (Als-Nielsen 2003Djulbegovic 2000Estellat 2012Katz 2006Lathyris 2010), active comparators in inferior doses (Rochon 1994Safer 2002) or inappropriate administration of the drugs (Johansen 1999). Or, industry sponsored studies may be biased in the coding of events and their data analysis (Furukawa 2004Psaty 2008Psaty 2010). Industry and its sponsored investigators also may selectively report favorable outcomes, fail to publish whole studies with unfavorable results, or publish studies with favorable results multiple times (Chan 2004Dwan 2008Gøtzsche 2011McGauran 2010Melander 2003Rising 2008Vedula 2009). While such biases in analyses and reporting have been documented in a number of cases, the papers included in this review focused on comparisons of published studies. Therefore, we are unable to determine the extent to which selective analysis or reporting contribute to our findings.

The finding that industry sponsored studies are more likely to have favorable conclusions could be explained by use of spin in conclusions (Boutron 2010). It should also be noted that some studies in the non-industry group likely had authors with conflicts of interest, which may have influenced their interpretation of study results (Stelfox 1998; Wang 2010) thereby diluting the measured effect of industry bias on study conclusions. Also, we coded studies as non-industry sponsored if they did not state who sponsored the study. As some of these studies were likely industry sponsored, this misclassification will have led to similar bias towards the null. In our sensitivity analyses, we excluded studies without sponsorship statements and did not see a change in results, but the confidence intervals were wide and did not exclude a possible bias towards the null.

Further evidence for industry bias stems from our comparison of studies sponsored by the manufacturer of the test treatment with those sponsored by the manufacturer of the control treatment. These studies had the advantage of comparing like with like, as they are restricted to specific drug classes or types of devices and have similar methodologies. Though limited to only three papers on drug trials, the findings show associations that are stronger than the comparison between industry and non-industry sponsored studies. These comparisons are restricted to drugs competing for the same market, which may put pressure on companies to influence outcomes to a greater degree than what is needed in placebo controlled trials to present the drug in a good light.

In sum, the industry bias associated with favorable results and conclusions may be mediated by factors other than traditional measures of the risk of bias (e.g. lack of concealment of allocation, blinding and drop-out) and sample size. This industry bias may be partially mediated by such factors as the choice of comparators, dosing and timing of comparisons, selective analysis, and selective reporting.

Quality of the evidence

The majority of included papers were regarded as having a high risk of bias. Many lacked information on study conduct and did not control for confounders that could influence the relationship. Nevertheless, we did identify nine papers with low risk of bias and analyses restricted to these papers actually strengthened the relationship between sponsorship and outcomes. In general, there is convincing and consistent evidence for the existence of an industry bias in studies; however, the evidence for device studies is not as strong as for drug studies. While papers, including studies of devices and other interventions, have been published in the surgical field (Cunningham 2007; Khan 2008; Leopold 2003; Roach 2008; Shah 2005; Yao 2007), the papers do not report separate data for device studies.

Potential biases in the review process

We did a comprehensive search, our methods were based on pre-specified criteria in a protocol as outlined in Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (Higgins 2011a) and our review has substantially increased the number of included papers from our previous review (Lexchin 2003). Nevertheless, there are some limitations. First, we decided only to include published papers. In our previous review (Lexchin 2003), we found problems with the completeness and quality of the data in conference abstracts and letters and therefore decided not to include them in this review. In our searches, we identified five conference abstracts and five letters that otherwise seemed to fit our inclusion criteria (Bond 2009; Djulbegovic 1999; Esquitin 2010; Higgins 2005; Koepp 1999; Mandelkern 1999; Thomas 2002; Vandenbroucke 2000; Wagena 2003; Wahlbeck 1999). Most were small (including a median of 30 studies, range 12 to 567 studies). Data from four papers could be included in a pooled analysis and gave similar findings for the association between sponsorship and study conclusions, RR: 1.57 (95% CI: 1.21 to 2.03), and RR: 1.32 (95% CI: 1.21 to 1.45) when they were added to the published papers (data available from authors on request). This makes publication bias unlikely to have influenced our results.

Second, our assessment of risk of bias in the included papers was not based on validated criteria similar to 'Risk of bias' assessment for clinical trials (Higgins 2011b). As no validated assessment tools exist for these type of papers, we developed our own criteria and included items similar to assessment tools for systematic reviews (Oxman 1991; Shea 2007).

Third, one item not included in our assessment of risk of bias in the papers was whether coders of outcomes were blinded to the sponsorship status of the studies. If these types of papers were undertaken by authors with a particular view on the drug industry, knowledge of sponsorship status could introduce bias in the assessment of whether outcomes were favorable, particularly for conclusions, as this is an outcome that is qualitative in nature. Some of the included papers were written by authors who had published multiple times in the area, and as such could be at increased risk of bias. These papers used coders who were both blinded and unblinded to the sponsorship status of the studies. The agreement in coding was high, suggesting a lack of bias (Als-Nielsen 2003; Bero 2007; Kjaergard 2002). Likewise, most of us (AL, JL, LB, SS) have published several times in the field and one of us (LB) is the author of four of the included papers (Bero 2007; Cho 1996; Rasmussen 2009; Rattinger 2009), which could have introduced bias. Because of the way data were presented in the papers, it was not possible to blind our data extraction process, so instead data extraction was undertaken by two of us with modest experience in the field and who were not authors of the original review or any of the included papers (AL, SS). Furthermore, our data extraction of outcomes did not involve any qualitative interpretation as we extracted actual numbers.

Fourth, if the papers included in this review included some of the same studies, their findings would not be independent. It was not possible to assess the potential overlap of studies as most papers did not provide a reference list of included studies. However, any overlap of included studies is likely to be very small and unimportant, as the disease and intervention topics of the included papers varied widely. 

Agreements and disagreements with other studies or reviews

Our results are in agreement with previous systematic reviews (Bekelman 2003; Lexchin 2003; Schott 2010; Sismondo 2008a), though the risk ratios for the associations are less than previous quantitative estimates. Previous reviews did not distinguish between favorable results or conclusions, but looked at the association between sponsorship and outcomes. Bekelman found OR 3.60 (95% CI: 2.63 to 4.91) and Lexchin OR 4.05 (95% CI: 2.98 to 5.51). Translated to odds ratios, we found 2.14 (95% CI: 1.70 to 2.71) for results and 2.67 (95% CI 2.02 to 3.53) for conclusions in our review. This difference could be due to chance or it could be because the earlier reviews also included pharmacoeconomic analyses, non-drug studies, letters and conference presentations. It is also possible that the degree of industry bias has diminished over time, for example with a decrease in reporting bias due to trial registration. However, we do not find it likely. First, a recent study found that reporting bias is also prevalent in registered trials (Mathieu 2009). Second, one of the most recent papers (Bourgeois 2010) sampled drug trials registered at clinicaltrials.gov and conducted between 2000 and 2006 and found OR: 4.50 (95% CI: 2.60 to 7.80) for results, suggesting that industry bias has not changed over time. 

Authors' conclusions

Implication for methodological research

Currently, the Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 acknowledges problems in relation to sponsorship, but does not recommend assessing industry sponsorship as a separate domain in the 'Risk of bias' assessment (Higgins 2011b). The assumption is that the influence of the sponsor will be mediated through the mechanisms of bias that are currently assessed, such as selective reporting of favorable outcomes. A Cochrane review that examined the association of sponsorship and selective outcome reporting bias (Dwan 2011) found uncertain evidence for the association; however, assessment of selective outcome reporting is complex and bias may be difficult to detect (Kirkham 2010). Some studies that have documented the extensive selective reporting of favorable outcomes have examined only industry sponsored studies (Rising 2008; Vedula 2009), thus making comparison with non-industry sponsored studies impossible.

Our data suggest that the more favorable outcomes in industry sponsored studies are mediated by factors other than those documented in the 'Risk of bias' assessment tool in Cochrane reviews. It has been suggested that industry bias should be regarded as a meta-bias, as industry sponsorship in itself is not a bias-producing process –  as for example lack of concealment of allocation is – but a risk factor for bias (Goodman 2011). However, the characteristics currently assessed in the standard risk of bias approach in Cochrane reviews likely do not capture the additional risk of bias in industry sponsored studies. For example, the Handbook states that design issues, such as dosage of comparators are not issues of bias, but of generalizability. Yet, pharmacological interventions have dose-response curves, and testing drugs that are not in comparable places on their dose-response curves sets up a systematic, unfair and biased comparison (Safer 2002).

Consequently, our data suggest that industry sponsorship should be treated as bias-inducing and industry bias should be treated as a separate domain. There are many subtle mechanisms through which sponsorship may influence outcomes, and an assessment of sponsorship should therefore be used as a proxy for these mechanisms. Interestingly, the AMSTAR tool for methodological quality assessment of systematic reviews includes funding and conflicts of interest as a domain (Shea 2007). Methods for reporting, assessing and handling industry bias and other biases in future systematic reviews must be developed. Specifically, further methodological research should focus on how industry bias is handled in Cochrane reviews.

Acknowledgements

We thank Bryan Sandlund for help with developing the protocol, Sarah Chapman (Trial Search Co-ordinator) at the Methodology Review Group for developing the search strategy and running the database searches and Asbjørn Hróbjartsson for discussion of methodological issues. We thank the authors of the included papers for sharing their raw data.

Data and analyses

Download statistical data

Comparison 1. Results: Industry sponsored versus non-industry sponsored studies
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Number of studies with favorable efficacy results141588Risk Ratio (M-H, Fixed, 95% CI)1.32 [1.21, 1.44]
2 Number of studies with favorable harms results3561Risk Ratio (M-H, Fixed, 95% CI)1.87 [1.54, 2.27]
Analysis 1.1.

Comparison 1 Results: Industry sponsored versus non-industry sponsored studies, Outcome 1 Number of studies with favorable efficacy results.

Analysis 1.2.

Comparison 1 Results: Industry sponsored versus non-industry sponsored studies, Outcome 2 Number of studies with favorable harms results.

Comparison 2. Results: Industry sponsorship by test treatment company versus sponsorship by comparator treatment company
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Number of studies with favorable test treatment efficacy results2131Risk Ratio (M-H, Fixed, 95% CI)4.64 [2.08, 10.32]
Analysis 2.1.

Comparison 2 Results: Industry sponsorship by test treatment company versus sponsorship by comparator treatment company, Outcome 1 Number of studies with favorable test treatment efficacy results.

Comparison 3. Conclusions: industry sponsored versus non-industry sponsored studies
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Number of studies with favorable conclusions213941Risk Ratio (IV, Random, 95% CI)1.31 [1.20, 1.44]
Analysis 3.1.

Comparison 3 Conclusions: industry sponsored versus non-industry sponsored studies, Outcome 1 Number of studies with favorable conclusions.

Comparison 4. Conclusions: Industry sponsorship by test treatment company versus sponsorship by comparator treatment company
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Number of studies with favorable test treatment conclusions3154Risk Ratio (M-H, Fixed, 95% CI)5.90 [2.79, 12.49]
Analysis 4.1.

Comparison 4 Conclusions: Industry sponsorship by test treatment company versus sponsorship by comparator treatment company, Outcome 1 Number of studies with favorable test treatment conclusions.

Comparison 5. Risk of bias: industry sponsored versus non-industry sponsored studies
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Number of studies with low risk of bias from sequence generation3487Risk Ratio (IV, Random, 95% CI)0.85 [0.52, 1.41]
2 Number of studies with low risk of bias from concealment of allocation101311Risk Ratio (IV, Random, 95% CI)1.09 [0.86, 1.38]
3 Number of studies with low risk of bias from blinding91216Risk Ratio (IV, Random, 95% CI)1.32 [1.05, 1.65]
4 Number of studies with low risk of bias from loss to follow-up2118Risk Ratio (M-H, Fixed, 95% CI)0.98 [0.84, 1.16]
Analysis 5.1.

Comparison 5 Risk of bias: industry sponsored versus non-industry sponsored studies, Outcome 1 Number of studies with low risk of bias from sequence generation.

Analysis 5.2.

Comparison 5 Risk of bias: industry sponsored versus non-industry sponsored studies, Outcome 2 Number of studies with low risk of bias from concealment of allocation.

Analysis 5.3.

Comparison 5 Risk of bias: industry sponsored versus non-industry sponsored studies, Outcome 3 Number of studies with low risk of bias from blinding.

Analysis 5.4.

Comparison 5 Risk of bias: industry sponsored versus non-industry sponsored studies, Outcome 4 Number of studies with low risk of bias from loss to follow-up.

Comparison 6. Concordance between study results and conclusions: industry sponsored versus non-industry sponsored studies
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Number of studies with concordant study results and conclusions5667Risk Ratio (IV, Random, 95% CI)0.84 [0.70, 1.01]
Analysis 6.1.

Comparison 6 Concordance between study results and conclusions: industry sponsored versus non-industry sponsored studies, Outcome 1 Number of studies with concordant study results and conclusions.

Comparison 7. Subgroup analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Number of studies with favorable efficacy results141588Risk Ratio (M-H, Fixed, 95% CI)1.32 [1.21, 1.44]
1.1 High risk of bias11962Risk Ratio (M-H, Fixed, 95% CI)1.19 [1.06, 1.33]
1.2 Low risk of bias3626Risk Ratio (M-H, Fixed, 95% CI)1.53 [1.32, 1.78]
2 Number of studies with favorable conclusions213941Risk Ratio (IV, Random, 95% CI)1.31 [1.20, 1.44]
2.1 High risk of bias173062Risk Ratio (IV, Random, 95% CI)1.26 [1.14, 1.39]
2.2 Low risk of bias4879Risk Ratio (IV, Random, 95% CI)1.54 [1.24, 1.91]
3 Number of studies with favorable conclusions213941Risk Ratio (IV, Random, 95% CI)1.30 [1.18, 1.42]
3.1 Drug studies213821Risk Ratio (IV, Random, 95% CI)1.31 [1.19, 1.44]
3.2 Device studies2120Risk Ratio (IV, Random, 95% CI)1.09 [0.82, 1.45]
4 Number of studies with favorable efficacy results141588Risk Ratio (M-H, Fixed, 95% CI)1.32 [1.21, 1.44]
4.1 Specific treatments or diseases121143Risk Ratio (M-H, Fixed, 95% CI)1.28 [1.15, 1.43]
4.2 Mixed domain2445Risk Ratio (M-H, Fixed, 95% CI)1.41 [1.19, 1.66]
5 Number of studies with favorable conclusions213941Risk Ratio (IV, Random, 95% CI)1.31 [1.20, 1.44]
5.1 Specific treatments or diseases162774Risk Ratio (IV, Random, 95% CI)1.34 [1.19, 1.51]
5.2 Mixed study domain51167Risk Ratio (IV, Random, 95% CI)1.26 [1.08, 1.47]
Analysis 7.1.

Comparison 7 Subgroup analysis, Outcome 1 Number of studies with favorable efficacy results.

Analysis 7.2.

Comparison 7 Subgroup analysis, Outcome 2 Number of studies with favorable conclusions.

Analysis 7.3.

Comparison 7 Subgroup analysis, Outcome 3 Number of studies with favorable conclusions.

Analysis 7.4.

Comparison 7 Subgroup analysis, Outcome 4 Number of studies with favorable efficacy results.

Analysis 7.5.

Comparison 7 Subgroup analysis, Outcome 5 Number of studies with favorable conclusions.

Comparison 8. Sensitivity analysis
Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size
1 Number of studies with favorable efficacy results, sponsorship recoded5517Risk Ratio (M-H, Fixed, 95% CI)1.50 [1.27, 1.76]
2 Number of studies with favorable conclusions, sponsorship recoded7951Risk Ratio (IV, Random, 95% CI)1.26 [1.06, 1.50]
3 Number of studies with low risk of bias from sequence generation, sponsorship recoded2249Risk Ratio (M-H, Fixed, 95% CI)1.05 [0.77, 1.44]
4 Number of studies with low risk of bias from concealment of allocation, sponsorship recoded7663Risk Ratio (M-H, Fixed, 95% CI)1.17 [0.90, 1.52]
5 Number of studies with low risk of bias from blinding, sponsorship recoded5425Risk Ratio (M-H, Fixed, 95% CI)1.54 [1.23, 1.94]
6 Number of studies with favorable efficacy results, analysis adjusted for confounders2 Odds Ratio (Fixed, 95% CI)3.86 [1.93, 7.70]
7 Number of studies with favorable conclusions, analysis adjusted for confounders3 Odds Ratio (Fixed, 95% CI)4.15 [2.40, 7.19]
8 Number of studies with favorable efficacy results, random-effects model141588Risk Ratio (IV, Random, 95% CI)1.26 [1.12, 1.41]
9 Number of studies with favorable harms results, random-effects model3561Risk Ratio (IV, Random, 95% CI)1.87 [1.54, 2.27]
10 Number of studies with favorable test treatment efficacy results, random-effects model2131Risk Ratio (IV, Random, 95% CI)3.88 [1.32, 11.41]
11 Number of studies with favorable test treatment conclusions, random-effects model3154Risk Ratio (IV, Random, 95% CI)5.92 [2.80, 12.54]
12 Number of studies with low risk of bias from loss to follow-up, random-effects model2118Risk Ratio (IV, Random, 95% CI)0.99 [0.84, 1.15]
Analysis 8.1.

Comparison 8 Sensitivity analysis, Outcome 1 Number of studies with favorable efficacy results, sponsorship recoded.

Analysis 8.2.

Comparison 8 Sensitivity analysis, Outcome 2 Number of studies with favorable conclusions, sponsorship recoded.

Analysis 8.3.

Comparison 8 Sensitivity analysis, Outcome 3 Number of studies with low risk of bias from sequence generation, sponsorship recoded.

Analysis 8.4.

Comparison 8 Sensitivity analysis, Outcome 4 Number of studies with low risk of bias from concealment of allocation, sponsorship recoded.

Analysis 8.5.

Comparison 8 Sensitivity analysis, Outcome 5 Number of studies with low risk of bias from blinding, sponsorship recoded.

Analysis 8.6.

Comparison 8 Sensitivity analysis, Outcome 6 Number of studies with favorable efficacy results, analysis adjusted for confounders.

Analysis 8.7.

Comparison 8 Sensitivity analysis, Outcome 7 Number of studies with favorable conclusions, analysis adjusted for confounders.

Analysis 8.8.

Comparison 8 Sensitivity analysis, Outcome 8 Number of studies with favorable efficacy results, random-effects model.

Analysis 8.9.

Comparison 8 Sensitivity analysis, Outcome 9 Number of studies with favorable harms results, random-effects model.

Analysis 8.10.

Comparison 8 Sensitivity analysis, Outcome 10 Number of studies with favorable test treatment efficacy results, random-effects model.

Analysis 8.11.

Comparison 8 Sensitivity analysis, Outcome 11 Number of studies with favorable test treatment conclusions, random-effects model.

Analysis 8.12.

Comparison 8 Sensitivity analysis, Outcome 12 Number of studies with low risk of bias from loss to follow-up, random-effects model.

Appendices

Appendix 1. Search strategy

1. Drug Industry/

2. ((drug$ or pharmaceutical$ or device$ or for-profit) adj (industr$ or company or companies or manufacturer$ or organisation$ or organization$ or agency or agencies)).ti,ab. 

3. private industr$.ti,ab. 

4. (industr$ or nonindustr$ or non-industr$).ti,ab. 

5. 1 or 2 or 3 or 4 

6. Conflict of interest/ 

7. Financial support/ 

8. Research support as topic/ 

9. (funded or funding or sponsor$ or support$ or financ$ or involvement).ti,ab. 

10. "competing interest$".ti,ab. 

11. or/6-10 

12. 5 and 11 

13. Publication bias/ 

14. "Bias (Epidemiology)"/ 

15. bias$.ti,ab. 

16. or/13-15 

17. 12 and 16 

18. Treatment outcome/ 

19. "Outcome Assessment (Health Care)"/ 

20. (outcome$ or findings).ti,ab. 

21. or/18-20 

22. (favor$ or favour$ or positive or significan$ or beneficial or benefit$ or effective or effectual or efficacious).ti,ab. 

23. (insignifican$ or nonsignifican$ or negative or adverse or ineffectiv$ or ineffectual or unfavorabl$ or unfavourabl$).ti,ab. 

24. 22 or 23 

25. 21 and 24 

26. 12 and 25 

27. ((favor$ or favour$ or positive or significan$ or insignifican$ or nonsignifican$ or negative or unfavorabl$ or unfavourabl$) adj result$).ti,ab. 

28. 12 and 27 

29. 17 or 26 or 28

Feedback

Feedback from Adam Jacobs, 20 December 2012

Summary

There are a number of problems with this review: primarily that the results do not support the conclusions drawn from them, and also that the authors did not account for publication bias. I have written a blogpost describing my concerns at http://dianthus.co.uk/cochrane-review-on-industry-sponsorship. [This is pasted below]

Many papers have been published that compare clinical trial publications sponsored by the pharmaceutical industry with those not sponsored by industry. Last week, the Cochrane Collaboration published a systematic review by Lundh et al of those papers. The stated objectives of the review were to investigate whether industry sponsored studies have more favourable outcomes and differ in risk of bias, compared with studies having other sources of sponsorship.

There are some rather extraordinary things about this review.

The most extraordinary thing is a high level of discordance between the results and the conclusions. This is a little odd, since one of the outcomes they investigated was whether industry studies were more prone to discordance between results and conclusions, so you’d have thought Lundh et al would understand the importance of making them match.

But nonetheless, they don’t seem to. The conclusions of the review state “our analyses suggest the existence of an industry bias”. In their results section, however, they investigated various items known to be associated with bias, such as randomisation and blinding. They found that industry studies had a lower risk of bias than non-industry studies. I’ve written before about bias in papers about bias, and this seems to be another classic example of the genre. This is disappointing in a Cochrane review. Cochrane reviews are supposed to be among the highest quality sources of evidence that there are, but this one falls a long way short.

It appears that they drew this conclusion because they found that industry sponsored trials were more likely to produce results or conclusions favourable to the sponsor’s product than independent trials (although that finding may not be as sound as they think it is, for reasons I’ll explain below). They therefore concluded that industry-sponsored trials must be biased, because they’re systematically different from independent trials. That does not make logical sense. Three explanations are possible: either industry trials are biased in favour of favourable results, independent trials are biased towards the null, or the two types of trial investigate systematically different questions. Any of those is possible, and they have not presented any evidence that allows us to distinguish between the possibilities. However, given that where they did measure bias, they found less bias in industry studies, the conclusion that the bias must be a result of industry sponsorship seems hard to support.

Another of Lundh et al’s conclusions was that industry-sponsored trials are more likely to have discordant results and conclusions, for example claiming that a result was favourable in the conclusions when the results don’t support that conclusion (I know, it’s hard to imagine anyone could do that, isn’t it?) This is stated as fact, despite the little drawback that their meta-analysis estimate of the difference between industry and non-industry studies did not reach statistical significance. Also, there is one study I happen to be aware of which would seem to be relevant to this analysis (Boutron et al 2010) as it investigated “spin” in conclusions, which seems to me to be exactly the same concept as discordance between results and conclusions. That study, for reasons not explained in the paper, was not included in their analysis. It can’t be because they didn’t know about it, as they cited it in their discussion (and, incidentally, misrepresented its results when they did so). Boutron et al found no significant difference between industry and non-industry studies in the prevalence of spin in conclusions, so if it had been included it could have weakened their results further.

I mentioned above that I was not totally convinced by their conclusion that industry-sponsored studies are more likely to have results favourable to the sponsor’s products than independent studies. Oddly enough, until I read this systematic review, I had taken that assertion as established fact. I have seen various papers that found that result, and had felt that the finding was robust. However, I now have my doubts.

Here’s why.

One of the big challenges for any systematic review is the problem of publication bias. This is the tendency of positive studies to be published and negative studies to be quietly forgotten. This is a big problem, because if you look at all published studies in a systematic review, you are actually looking at a biased subset of studies, usually those with positive results.

A good systematic reviewer will investigate the extent to which this is a problem. The Cochrane handbook, the instruction manual for Cochrane systematic reviews, recommends that reviewers investigate publication bias by means of funnel plots or statistical tests. The idea behind such methods is that large studies are likely to be published whatever the results, as so much has been invested in them that the final stage of publication is unlikely to be overlooked, whereas small studies may well be unpublished if they are negative, but are more likely to be published if they are positive. If you see a correlation between study size and effect size, with smaller studies showing larger effects than larger studies, that is strongly suggestive of publication bias. For those who are not familiar with these concepts, Wikipedia has a good explanation.

However, despite the recommendation in the Cochrane handbook that reviewers should investigate publication bias, Lundh et al seem to have largely overlooked it. They mention a small number of studies published only as conference abstracts or letters, and found they provided similar results to the main analysis, and concluded that publication bias was therefore unlikely. This is a very superficial examination of publication bias that falls well short of what should happen in a Cochrane review.

Fortunately, they present their data in full, so it is easy enough for anyone reading the review to do their own test for publication bias. So I did this for their primary analysis: comparing industry and non-industry studies for their probability of producing favourable results. The results are strongly indicative of publication bias. This is what the funnel plot looks like [available from http://dianthus.co.uk/cochrane-review-on-industry-sponsorship].

As you can see, there is striking asymmetry here, with most small studies (those towards the bottom: the y scale is actually the reciprocal of the standard error of the relative risk, but this is strongly related to study size) having much larger effects than larger studies, and no small studies showing smaller effects. This is very strongly suggestive of publication bias. I also did a statistical test for publication bias (the Egger test, one of those recommended in the Cochrane handbook), with a regression coefficient for effect size on standard error of 2.3, which was statistically significant at P = 0.026.

So there is clear evidence that these results were subject to publication bias. It is therefore highly likely that their estimate of the difference between industry and non-industry studies was overstated. Maybe there isn’t really a difference at all. It’s very hard to tell, when the literature is not complete.

I could go on, as there are other flaws in the paper, but I think that’s long enough for one blog post. So to sum up, this Cochrane review had methods that fell short of what is expected for Cochrane reviews. Lundh et al found that industry sponsored studies, when assessed using well established measures of bias, were less likely to be biased than independent studies, and yet drew the opposite conclusion, based on nothing but speculation. This, in a study which investigated discordance between results and conclusions, is bizarre. Their main finding, that industry sponsored studies were more likely to generate favourable results than independent studies, appears to have been affected by publication bias, which makes it considerably less reliable than Lundh et al claim.

I am normally a great fan of the Cochrane Collaboration, which usually produces some of the best quality syntheses of clinical evidence that you will ever find. To see such a biased review from them is deeply disappointing.

I have modified the conflict of interest statement below to declare my interests: I run a commercial company that provides consultancy services to clinical researchers from the pharmaceutical industry and to academic researchers, but more often to the former.

Reply

We thank Adam Jacobs for the interest in our paper and would like to respond. The comment deals with three issues: the exclusion of the Boutron paper, the use of the term industry bias and possible publication bias.

First, Jacobs believes we should have included the Boutron paper (1). This paper was identified in our search, but excluded from our review because it did not meet the inclusion criteria pre-specified in our protocol. The Boutron paper deals with spin in conclusion of trials with statistically nonsignificant results. Spin can be focusing on results of subgroups or focusing on secondary outcomes in the discussion. This is something very different from concordance between results and conclusions investigated in our review (i.e. whether the conclusions agreed with the study results). Also, we did not misrepresent the Boutron paper as Jacobs suggests. Jacobs states that, “Boutron et al found no significant difference between industry and non-industry studies in the prevalence of spin in conclusions.” Boutron does not make this claim. In fact Boutron et al. writes “Our results are consistent with those of other related studies showing a positive relation between financial ties and favourable conclusions stated in trial reports.”

Second, Jacobs misstates our results by suggesting that we found that industry studies had a lower risk of bias than non-industry studies. We did not find a difference, except in relation to blinding, which we discuss further in our review. Furthermore, Jacobs actually seems to agree with our conclusion that the industry bias may be due to factors other than the traditional risks of bias that are usually assessed, e.g., randomization and blinding. That is just our point. For example, he states that the industry studies may “investigate systemically different questions”. We mention this possibility in our Discussion section. Based on the available external evidence (2,3), we believe that a plausible explanation for the more favorable results and conclusions in industry studies is that industry studies may be biased in design, conduct, analysis and reporting.

Third, Jacobs suggests that our results may be due to publication bias (i.e. that papers finding no difference in favorable outcomes between industry sponsored versus non-industry sponsored studies are not published) and criticizes us for not assessing publication bias using a funnel plot. However, as the Cochrane Handbook states (4) (section 10.4), there may be various reasons for funnel plot asymmetry, publication bias being one of them. Other reasons are true heterogeneity and risk of bias in the primary material being used in the meta-analysis. The included papers investigated many different drugs and devices and it is likely that any ‘industry bias’ may be different between the various types of treatments. Due to the anticipated heterogeneity of these papers, we did not assess publication bias using a funnel plot, as such a plot would be difficult to interpret as noted in the Cochrane Handbook. Instead, we included conference abstracts and letters in an additional analysis and found it had no impact.

To argue for his case about publication bias Jacobs presents a funnel plot of analysis 1.1 on the association between sponsorship and favorable results. Based on this funnel plot four small studies are outliers to the right and should provide evidence of bias. However, three of these studies were related to specific drugs (glucosamine, nicotine replacement therapy and antipsychotics) and the fourth study dealt with psychiatric research, a field where biased industry research has been well documented (5,6). So the study domain may explain the difference. Also three of the four studies had high risk of bias, which could also explain the findings and if we restrict the analysis to studies of low risk of bias (analysis 7.1) the results are less heterogeneous. Last, even if we assume publication bias to be present, to exclude the four papers from our analysis (the ones that should provide evidence for publication bias) has no impact on our analysis (RR 1.27 (95% CI: 1.16 to 1.39 ) instead of 1.32 (95% CI: 1.21 to 1.44 )). A similar plot for conclusions (analysis 3.1) has two studies as outliers and excluding these studies from the analysis has no impact (RR 1.27 (95% CI: 1.16 to 1.38) instead of RR 1.31 (95% CI: 1.20 to 1.44)). We therefore completely disagree with Jacob’s statement that “there is clear evidence that these results were subject to publication bias. It is therefore highly likely that their estimate of the difference between industry and non-industry studies was overstated. Maybe there isn’t really a difference at all”. On the contrary, our findings that industry sponsorship leads to more favorable results and conclusions, are very robust.

In sum, none of the comments provided by Jacobs have any impact on our results.

Contributors

Andreas Lundh, Sergio Sismondo, Joel Lexchin and Lisa Bero

What's new

Last assessed as up-to-date: 30 September 2010.

DateEventDescription
30 April 2013AmendedIn the previous version, some of the analyses were by mistake not done according to the model prespecified in the protocol (e.g. inverse variance or Mantel–Haenszel). The analyses have now been changed. This had minor impact on some of the results and no impact on the conclusions.
25 March 2013Feedback has been incorporatedFeedback from Adam Jacobs and response from reviewers added.

Contributions of authors

Development of protocol (AL, JL, LB and SS); design of the search strategy (AL); initial screening of articles (AL and OAB); final selection of studies (all authors); data extraction (AL and SS); data analysis and interpretation of results (all authors); writing of manuscript (all authors).

Declarations of interest

Joel Lexchin, Lisa Bero and Sergio Sismondo are authors of the some of the previous reviews and included studies.

In 2007, Joel Lexchin was retained by a law firm representing Apotex to provide expert testimony about the effects of promotion on the sales of medications. From 2007 to 2008 he was retained as an expert witness by the Canadian federal government in its defense of a lawsuit challenging the ban on direct-to-consumer advertising of prescription drugs in Canada. In 2010 he was a consultant to a law firm acting for the family of a patient who died from an alleged side effect of a drug made by Allergan. He is also on the management group of Healthy Skepticism Inc. and is the chair of the Health Action International – Europe Association Board.

The authors have no other relevant interests. 

Sources of support

Internal sources

  • The Nordic Cochrane Centre, Copenhagen, Denmark.

    The author was personally salaried by his institution during the period of the review.

  • York University, Toronto, Canada.

    The author was personally salaried by his institutions during the period of the review.

  • Queen's University, Kingston, Canada.

    The second author was personally salaried by his institution during the period of the review.

  • University of California, San Francisco, USA.

    The author was personally salaried by her institution during the period of the review.

External sources

  • Canadian Institutes of Health Research, Canada.

    The work of Octavian A. Busuioc was supported by a grant from the Canadian Institutes of Health Research (#106892).

  • Julie von Müllens Foundation, Faculty of Health Sciences - University of Copenhagen and Kontorchef Gerhard Brønsteds Travel Grant, Denmark.

    Part of the work of Andreas Lundh was done during a visiting fellowship at University of California - San Francisco. Julie von Müllens Foundation, The Faculty of Health Sciences - University of Copenhagen and Kontorchef Gerhard Brønsteds Travel Grant paid for travel expenses and accommodation.

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Ahmer 2005

MethodsTo study the association between study support and outcome in randomized controlled trials (RCTs) of psychotropic drugs. All RCTs published in Acta Psychiatrica Scandinavica (APS), American Journal of Psychiatry (AJP), Archives of General Psychiatry (AGP) and British Journal of Psychiatry (BJP) from July 1998 to June 2003.
Data188 psychotropic drug RCTs (various comparators).
ComparisonsManufacturer support and no support.
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesDatabase and handsearch.
Control for bias?NoSubgroup analysis, but only of journal name.

Alasbali 2009

MethodsTo investigate the relationship between industry vs non-industry funded publications comparing the efficacy of topical prostaglandin analogs by evaluating the correspondence between the statistical significance of the publication’s main outcome measure and its abstract conclusions. Studies published from 1966 to November 2007.
Data39 reports of head-to-head comparisons of topical prostaglandins in ophthalmology (various study designs).
ComparisonsIndustry and non-industry funding.
OutcomesStudy conclusions, study results and concordance between study results and conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?UnclearNot clear which study designs and whether placebo controlled studies were included, cannot be replicated.
Adequate study inclusion process?UnclearThree assessors for data extraction, but unclear in relation to study inclusion.
Comprehensive search?YesMEDLINE and handsearching.
Control for bias?UnclearNot described.

Als-Nielsen 2003

MethodsTo explore whether the association between funding and conclusions in randomized drug trials reflects treatment effects or adverse events. All randomized trials included in eligible meta-analyses from a random sample of Cochrane reviews obtained in May 2001 (RCTs from 1971 to 2000).
Data370 drug RCTs (mixed comparisons).
ComparisonsFunding from non-profit organizations, not reported, both non-profit and for-profit organizations, and for-profit organizations.
OutcomesStudy conclusions, effect size and methodological quality (generation of randomization sequence, concealment of allocation and double blinding).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?YesOne assessor screened and two involved in final inclusion.
Comprehensive search?YesIdentification via Cochrane reviews.
Control for bias?YesLogistic regression adjusting for treatment effect, adverse events, and other potentially confounding trial variables (methodological quality, sample size, whether preset sample size was estimated and reached, meta-analysis, year of publication, and journal impact factor). Adjusted for treatment effect and double blinding in final model.

Barden 2006

MethodsTo study if industry sponsored trials yield a better result than trials not sponsored by industry, and if a particular drug would perform better as the test drug in trials funded by its manufacturer and worse as the comparator drug in trials funded by a competitor. RCTs from published systematic reviews in acute pain and migraine (reviews from 1999 to 2004).
Data176 acute pain or migraine drug RCTs (active comparator or placebo controlled).
ComparisonsIndustry versus non-industry and manufacturer versus competitor funding.
OutcomesEffect size and methodological quality (Jadad score, 0-5 point scale).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?YesFrom Cochrane reviews, seems more than one assessor was used.
Comprehensive search?YesIdentification via Cochrane reviews.
Control for bias?NoNo control for bias.

Bero 2007

MethodsTo examine the associations between research funding source, study design characteristics aimed at reducing bias, and other factors that potentially influence results and conclusions in randomized controlled trials of statin–drug comparisons. All statin RCTs with active comparators from January 1999 to May 2005.
Data192 statin RCTs (active comparators).
ComparisonsIndustry, none disclosed/no funding and government/private non-profit funding.
OutcomesStudy results, study conclusions, methodological quality (concealment of allocation, blinding and follow-up) and concordance between study results and conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?YesTwo or more assessors included studies.
Comprehensive search?YesMEDLINE and references.
Control for bias?YesMultivariate logistic regression analysis. Final model controlled for journal Impact Factor, sample size and blinding.

Bhandari 2004

MethodsTo study the association between industry funding and the statistical significance of results in recently published medical and surgical trials. RCTs from January 1999 to June 2001 in 8 leading surgical journals (Journal of Bone and Joint Surgery [American and British volumes], Clinical Orthopaedics and Related Research, Acta Orthopaedica Scandinavica, Annals of Surgery, American Journal of Surgery, Plastic and Reconstructive Surgery and Journal of Neurosurgery) and 5 medical journals (Lancet, BMJ, JAMA, Annals of Internal Medicine and New England Journal of Medicine).
Data332 RCTs of drug, surgery, and other types of interventions (no description of comparisons).
ComparisonsIndustry-for-profit, not-for-profit and undeclared funding.
OutcomesStudy results and methodological quality (Detsky quality index, 0-21 point scale).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesHandsearch and MEDLINE used.
Control for bias?YesMultivariate logistic regression with adjustment for sample size, study quality and type of intervention.

Booth 2008

MethodsTo describe trends in methodology and reporting of RCTs, in addition to sponsorship, outcomes, and authors’ interpretation of results. All RCTs of systemic therapy in breast, colorectal cancer, and non-small-cell lung cancer published during three decades (1975 through 2004) in:Journal of Clinical Oncology, Journal of the National Cancer Institute, Cancer Treatment/Chemotherapy Reports, New England Journal of Medicine, Lancet, and JAMA.
Data321 drug RCTs (active comparators and placebo controlled).
ComparisonsFor-profit/mixed, non-profit and not known funding.
OutcomesStudy results, study conclusions and effect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesDatabase and handsearch.
Control for bias?YesMultivariate logistic regression, final model controlled for time to event, effect size and P value.

Bourgeois 2010

MethodsTo describe characteristics of drug trials listed in ClinicalTrials.gov and examine whether the funding source of these trials is associated with favorable published outcomes. Clinical trials registered from 2000 to 2006 and published up to 2010.
Data546 clinical trials of cholesterol-lowering drugs, antidepressants, antipsychotics, proton-pump inhibitors and vasodilators (active or placebo controlled).
ComparisonsIndustry, government and non-profit/non-federal (with or without industry contributions) funding.
OutcomesStudy results.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?YesTwo assessors independently carried out the literature search and disagreements were resolved by consensus.
Comprehensive search?YesFour databases, trial registries and contact to investigators and companies.
Control for bias?YesPost hoc multivariate logistic regression analysis to assess the association between funding source and trial outcome, while controlling for other trial characteristics (drug class, approval status of indication, study phase, multicenter status, anticipated sample size, age of study population, comparator type, and length of study).

Brown 2006

MethodsTo evaluate the trends in the source of funding for gastrointestinal clinical research during the period from 1992 to 2002–2003; to determine whether the source of study funding predicted the likelihood that a study would publish results that favor the drug or device being tested; and to determine whether differences exist in the methodologic quality of the investigational study methods used in studies funded by private industry versus other sources. Clinical trials published in 4 gastrointestinal journals (Gastroenterology, The American Journal of Gastroenterology, Hepatology, and Gastrointestinal Endoscopy).
Data450 clinical trials of drugs and devices in gastroenterology (active or placebo controlled).
ComparisonsPrivate industry sponsored, federal/state government sponsored, national society/non-profit agency sponsored and not specified.
OutcomesStudy conclusions and methodological quality (Brown score, 0 to 5 point scale multiplied by 100).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesHandsearching of journals.
Control for bias?NoNo control for bias.

Buchkowsky 2004

MethodsTo characterize clinical trial funding, reporting, and sources; investigate author-industry affiliation; and describe clinical outcome trends over time. Random papers from January 1981 to December 2000 from Annals of Internal Medicine, BMJ,JAMA,Lancet and New England Journal of Medicine.
Data500 clinical drug trials (drug versus placebo, active comparator or non-drug comparator).
ComparisonsIndustry, mixed, non-industry and not stated funding.
OutcomesStudy conclusions
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesHandsearching of journals.
Control for bias?UnclearInvestigates choice of comparators over time, might have assessed other sources of bias.

Chard 2000

MethodsTo assess the published research base for interventions for osteoarthritis of the knee, and to identify areas in need of further research. Studies from 1950 to 1998.
Data930 studies of different interventions (various study designs with various comparators).
ComparisonsCommercial, government and not stated funding.
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?NoOne assessor on all studies and one on 10% sample, but only 87% agreement indicating two needed for all studies.
Comprehensive search?YesMEDLINE, EMBASE, BIDS, The Cochrane Library, previous reviews and experts contacted.
Control for bias?NoNo control for bias.

Cho 1996

MethodsTo compare the quality, relevance, and structure of drug studies published in symposium proceedings that are sponsored by drug companies with 1) articles from symposia with other sponsors and 2) articles in the peer reviewed parent journals of symposium proceedings; and to study the relation between drug company sponsorship and study outcome. Random selection of symposia from 625 symposia that had been identified for a previous study.
Data127 drug studies (various study designs with various comparators).
ComparisonsDrug company support and no support.
OutcomesStudy conclusions and methodological quality (Cho scale 0-1 point).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?UnclearNot clear enough to replicate how symposia were chosen and how matching papers were chosen.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesComprehensive search within their own database.
Control for bias?YesSubgroup analysis of study design.

Clifford 2002

MethodsTo examine the relationship between funding source, trial outcome and reporting quality;100 RCTs from Annals of Internal Medicine, BMJ, JAMA, Lancet, New England Journal of Medicine. From January 1999 to October 2000 with 20 RCTs/journal.
Data100 drug RCTs (various comparators).
ComparisonsEntirely industry, entirely not-for-profit, mixed and not reported funding.
OutcomesStudy results, methodological quality (Jadad score, 0-5 point scale and concealment of allocation).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesHandsearching of journals.
Control for bias?NoNo evidence of risk of bias assessment.

Crocetti 2010

MethodsTo assess the risk of bias among pediatric RCTs reported in 8 high-impact journals (5 pediatric and 3 general medical) from July 2007 to June 2008.
Data146 pediatric drug, behavioral/educational and nutritional RCTs (various comparators)
ComparisonsGovernment, industry, internal hospital grant, multiple sources, none and private foundation funding.
OutcomesMethodological quality (sequence generation; allocation concealment; masking of participants, personnel, and outcome assessors; incomplete outcome data reporting; selective outcome reporting; and other sources of bias).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE search of selected journals.
Control for bias?YesMultivariate logistic regression to test for an association between the presence of a high risk of bias according to domain and the independent variables of funding source, intervention type, author number, and trial registration status.

Davidson 1986

MethodsAn analysis of the results of clinical trials according to funding source. Clinical trials from 1984 in New England Journal of Medicine, Annals of Internal Medicine,the American Journal of Medicine,Archives of Internal Medicine, and the Lancet.
Data107 drug and non-drug clinical trials (various comparators).
ComparisonsPharmaceutical support and general support.
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?NoSingle assessor.
Comprehensive search?YesJournals handsearched.
Control for bias?NoControl for bias seems unlikely to have been done.

Davis 2008

MethodsThe influence of several potentially biasing factors (e.g. industry support, extrapyramidal side effects) on efficacy of studies comparing second-generation antipsychotic with first-generation drugs. Dataset from previously published meta-analysis (search from 1953 to 2002).
Data124 RCTs of second-generation antipsychotics versus first-generation antipsychotics.
ComparisonsIndustry and non-industry funding.
OutcomesEffect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesComprehensive database search including search for unpublished data.
Control for bias?UnclearCarried out various sensitivity analysis, but not clear whether they assessed bias in relation to funding and effect size.

Djulbegovic 2000

MethodsTo evaluate whether the uncertainty principle was upheld, comparison of the number of studies favoring experimental treatments over standard ones according to the source of funding. All RCTs for multiple myeloma from 1996 to 1998.
Data136 multiple myeloma drug RCTs (various comparators).
ComparisonsCommercial and public funding.
OutcomesStudy conclusions and methodological quality (Jadad score, 0-5 point scale).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?NoSeems only one author involved in study inclusion.
Comprehensive search?YesUsing the Cochrane search strategy to identify trials.
Control for bias?YesControlled for types of comparator (active versus placebo/no treatment).

Etter 2007

MethodsTo assess whether source of funding affected the results of trials of nicotine replacement therapy for smoking cessation. RCTs from 1979 to 2003 identified from Cochrane review.
Data105 RCTs of nicotine replacement therapy (gum or patch versus placebo or no treatment).
ComparisonsIndustry/mixed and non-industry/not acknowledged funding.
OutcomesStudy results and effect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?YesFrom Cochrane review, seems more than one assessors was used.
Comprehensive search?YesIdentification via Cochrane review.
Control for bias?YesMultivariate logistic regression with adjustment for sample size.

Finucane 2004

MethodsTo evaluate the association between funding and findings of pharmaceutical research presented at an annual meeting of a clinically oriented US medical professional society.
Data48 presentations of drug studies (observational studies, RCTs and other study designs).
ComparisonsIndustry supported and not industry supported.
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?UnclearUnclear what "any abstract that reported results about effectiveness or safety of drugs" means. Not clear which study designs and whether reviews were included.
Adequate study inclusion process?YesSeems likely that two assessors were used.
Comprehensive search?YesComprehensive search within conference.
Control for bias?YesSubgroup analysis of study design.

Freemantle 2000

MethodsTo assess whether specific pharmacological characteristics of alternative antidepressants resulted in altered efficacy compared to that of selective serotonin reuptake inhibitors (SSRI) in the treatment of major depression. All RCTs of SSRI versus alternative antidepressants (search from 1966 to 1997).
Data105 SSRI versus alternative antidepressant RCTs.
ComparisonsSponsor and not sponsor.
OutcomesEffect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE, EMBASE, references and reviews.
Control for bias?NoNo assessment of bias in relation to funding and effect size.

Gartlehner 2010

MethodsThe objective of this study was to determine the effect of industry bias in a systematically reviewed sample of head-to-head trials. Trials of SSRI head-to-head comparisons from 1993 to 2005.
Data29 SSRI RCTs of head-to-head comparisons.
ComparisonsSponsor and not sponsor.
OutcomesStudy results and effect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?YesTwo assessors included studies.
Comprehensive search?YesMEDLINE, EMBASE, The Cochrane Library, the International Pharmaceutical Abstracts database, references and reviews and letters to the editor. In addition, the Center for Drug Evaluation and Research database to identify unpublished research submitted to the US Food and Drug Administration (FDA).
Control for bias?YesSensitivity analysis based on definition of funding.

Halpern 2005

MethodsTo determine whether there is a difference in average statistical power between pharmacoepidemiologic studies of anti-retroviral adverse drug effects (ADEs) sponsored by for-profit versus non-profit organizations (drugs approved from 1987 to 1999 and published until 2002).
Data48 pharmacoepidemiological studies of adverse effects of ani-retroviral drugs.
ComparisonsNon-profit, for-profit, charity/institution, none or unable to determine funding.
OutcomesStudy results (harms).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?NoOne assessor only.
Comprehensive search?YesMEDLINE, EMBASE and reference lists.
Control for bias?NoNo control for bias.

Heres 2006

MethodsTo review the results of head-to-head studies of second-generation antipsychotics funded by pharmaceutical companies to determine if a relationship exists between the sponsor of the trial and the drug favored in the study’s overall outcome. All head-to-head trials of second-generation antipsychotics from 1997 to 2005.
Data42 head-to-head RCTs of second-generation antipsychotics.
ComparisonsIndustry only (sponsor of test drug or comparator).
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?NoMEDLINE and screen of selected conference proceedings. Sample of conference proceedings limited to 1999-2004, which may introduce bias due to differences in approval dates for the different drugs.
Control for bias?YesSensitivity analysis of peer-reviewed trials only.

Jefferson 2009

MethodsTo explore the relation between study concordance, take home message, funding, and dissemination of comparative studies assessing the effects of influenza vaccines. Studies of various designs from 1961 to 2006.
Data274 studies of influenza vaccine versus placebo/no treatment.
ComparisonsGovernment/private/unfunded, industry/mixed and not stated funding.
OutcomesStudy conclusions, methodological quality (Cochrane risk of bias) and concordance between study results and conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?YesTwo assessors included studies.
Comprehensive search?YesMEDLINE, EMBASE, The Cochrane Library, web, and likely references and previous reviews since it is based on Cochrane reviews.
Control for bias?YesSensitivity analysis based on definition of funding and regression analysis of various factors.

Jones 2010

MethodsTo compare the quality of publicly or privately funded randomized controlled trials. Trials included in Cochrane reviews on hypertension and preterm labour.
Data105 drug trials (mixed comparisons).
ComparisonsCommercial, mixed and non-commercial.
OutcomesMethodological quality (selection bias, performance bias, detection bias and attrition).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesBased on searches from Cochrane reviews.
Control for bias?NoNo control for bias.

Kelly 2006

MethodsTo investigate the relationship between industry support and study outcome in the general psychiatric literature. Clinical studies from 1992 and 2002 in American Journal of Psychiatry, Archives of General Psychiatry, and Journal of Clinical Psychopharmacology.
Data301 psychiatric drug studies (mixed comparisons).
ComparisonsNon-industry and industry (sponsor of test drug or comparator) funding.
OutcomesStudy results, study conclusions and concordance between study results and conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesSample of journals.
Control for bias?YesExplanatory analysis of various mediating variables.

Kemmeren 2001

MethodsTo evaluate quantitatively articles that compared effects of second- and third-generation oral contraceptives on risk of venous thrombosis. Cohort and case control studies from 1995 to 2000.
Data12 cohort and case control studies of second- versus third-generation oral contraceptives.
ComparisonsIndustry and non-industry funding.
OutcomesStudy results and effect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE, reviews, relevant papers and experts.
Control for bias?NoMultiple regression used, but not for the association between funding and results or effect size.

Kjaergard 2002

MethodsTo assess the association between competing interests and authors' conclusions. RCTs published in BMJ 1997 to 2001.
Data159 RCTs of mixed interventions (various comparators).
ComparisonsProfit, non-profit, non-profit and profit, non-profit and free drug, free drug only and no funding/not stated.
OutcomesStudy conclusions and methodological quality (sequence generation, concealment of allocation and blinding).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?NoOnly one assessor included studies.
Comprehensive search?YesMEDLINE journal search.
Control for bias?YesRegression analysis for potential confounders.

Liss 2006

MethodsTo determine whether drug studies in the pulmonary/allergy literature also demonstrate a publication bias towards more favorable results when a pharmaceutical company funds the study. Primary research studies of drug interventions published in Allergy, American Journal of Respiratory and Critical Care Medicine, Annals of Allergy Asthma and Immunology, Chest, European Respiratory Journal, Journal of Allergy and Clinical Immunology, Respiratory Medicine, and Thorax in 2002 to 2003.
DataStudies of nasal or oral inhaled corticosteroids, long- or short-acting bronchodilators, and leukotriene receptor antagonists (various designs and comparisons).
ComparisonsPharmaceutically and not pharmaceutically funded.
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?NoOnly one assessor included studies.
Comprehensive search?YesHandsearch of journals indirectly described.
Control for bias?NoNo control for bias.

Lubowitz 2007

MethodsTo compare outcomes (and levels of evidence) between published Autologous Chondrocyte Implantation outcome studies that were commercially funded and studies that were not commercially funded. Clinical studies from 1994 to 2005.
Data23 studies of chondrocyte implantation (various designs and comparisons).
ComparisonsCommercially funded and not commercially funded.
OutcomesEffect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?NoMEDLINE only, time period not stated and few search terms used.
Control for bias?NoNo control for bias.

Lynch 2007

MethodsTo test the following hypotheses regarding orthopedic manuscripts submitted for review: (1) non-scientific variables, including receipt of commercial funding, affect the likelihood that a peer-reviewed submission will conclude with a report of a positive study outcome, and (2) positive outcomes and other, non-scientific variables are associated with acceptance for publication. Cohort of manuscripts submitted involving original research on the subject of adult hip or knee reconstruction to The Journal of Bone and Joint Surgery (American Volume) between January 2004 and June 2005.
Data209 studies of knee or hip surgery (various designs, interventions and comparisons).
ComparisonsCommercial, non-funded and noncommercial/philanthropic funding.
OutcomesStudy conclusions and methodological quality (Sackett scale, 0 to 100%).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesSample of papers via journal submission system.
Control for bias?NoNo control for bias.

Momeni 2009

MethodsTo investigate if plastic surgical trials with industry-funding are more likely to be associated with statistically significant pro-industry findings. Trials in 4 plastic surgery journals (Plastic and Reconstructive Surgery, British Journal of Plastic Surgery, Annals of Plastic Surgery, and Aesthetic Plastic Surgery) from 1990 to 2005.
Data346 RCTs and controlled clinical trials (various designs, interventions and comparisons).
ComparisonsIndustry, public, university and not specified funding.
OutcomesStudy results.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesHandsearch of journals.
Control for bias?NoNo control for bias.

Moncrieff 2003

MethodsTo re-evaluate the evidence comparing clozapine with conventional antipsychotics and to investigate sources of heterogeneity. Trials from 1988 to 2001.
Data9 RCTs of clozapine versus conventional antipsychotics.
ComparisonsIndustry, other and not declared funding.
OutcomesStudy results and effect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?NoOnly one assessor included studies.
Comprehensive search?YesMEDLINE, EMBASE and Cochrane review.
Control for bias?NoUnivariate controlled for various predictors in relation to effect size only.

Montgomery 2004

MethodsTo analyze RCTs of second-generation antipsychotics in schizophrenia with respect to funding source (industry versus non-industry funding). RCTs from 1974 to 2002.
Data86 RCTs of 2nd generation antipsychotics versus other types (various comparisons).
ComparisonsIndustry and non-industry.
OutcomesStudy conclusions and methodological quality (Jadad score, 0-5 point scale).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE, PsychInfo and references.
Control for bias?NoNo control for bias.

Nieto 2007

MethodsTo evaluate differences between studies funded by the pharmaceutical manufacturer of the drug and those with no pharmaceutical funding regarding the findings and interpretation of adverse effects of inhaled corticosteroids. Studies from 1993 to 2002.
Data504 studies of inhaled corticosteroids (various study designs with various comparators).
ComparisonsPharmaceutical funded and not pharmaceutical funded.
OutcomesStudy results (harms), study conclusions (harms) and concordance between study results and conclusions (harms).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesSample of journals were identified by MEDLINE.
Control for bias?YesControlled for confounders using multivariate model.

Pengel 2009

MethodsTo examine the quality of reporting of RCTs in solid organ transplantation that were published 2004 to 2006.
Data332 trials in solid organ transplantation (mixed interventions and comparisons).
ComparisonsCommercial, nonprofit, mixed, no funding and not described.
OutcomesMethodological quality (concealment of allocation and Jadad score, 0-5 point scale).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE, EMBASE and The Cochrane Library.
Control for bias?NoNo control for bias.

Peppercorn 2007

MethodsTo evaluate the correlations between pharmaceutical company involvement, study design, and study outcome and to explore changes in these areas over time. Breast cancer trials of medical therapies that were published in the years 1993, 1998, and 2003 in 10 select English-language medical journals.
Data140 breast cancer drug trials (single arm studies and RCTs).
ComparisonsPharmaceutical studies versus non-pharmaceutical studies.
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesHandsearch and MEDLINE used.
Control for bias?NoOnly assessment of differences in study design in relation to funding.

Perlis 2005a

MethodsThe purpose was to determine the extent and impact of industry sponsorship conflicts of interest in dermatology research. Drug trials from Journal of Investigative Dermatology, Archives of Dermatology, British Journal of Dermatology, and Journal of the American Academy of Dermatology from 2000 to 2003.
Data179 RCTs of dermatological drugs (various comparators).
ComparisonsIndustry and non-industry funding.
OutcomesStudy conclusions and methodological quality (blinding and Jadad score, 0-5 point scale).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesSample of journals.
Control for bias?YesMultivariate regression analysis adjusted for conflict of interest, Jadad score, and sample size.

Perlis 2005b

MethodsTo study the extent and implications of industry sponsorship and financial conflicts of interest in psychiatric trials. Drug trials from the American Journal of Psychiatry, Archives of General Psychiatry, Journal of Clinical Psychiatry, andJournal of Clinical Psychopharmacology from 2001 to 2003.
Data397 psychiatric clinical drug trials (various comparators).
ComparisonsIndustry and non-industry funding.
OutcomesStudy results.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearNot sure if 3 assessors extracting data were involved in including studies.
Comprehensive search?YesMEDLINE and handsearch of journals.
Control for bias?YesLogistic regression adjusted for confounders.

Popelut 2010

MethodsTo examine financial sponsorship of dental implant trials, and to evaluate whether research funding sources affects the annual failure rate. Clinical trials from 1988 to 2005.
Data41 clinical trials of dental implants (single arm and active control).
ComparisonsIndustry, non-industry and unknown funding.
OutcomesEffect size.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?NoInclusion criteria reported, but not possible to decipher and seems subjective.
Adequate study inclusion process?YesTwo assessors included studies.
Comprehensive search?YesMEDLINE and handsearch.
Control for bias?YesControlled for confounders using multivariate analysis.

Rasmussen 2009

MethodsTo compare the prevalence of favorable results and conclusions among published reports of registered and unregistered RCTs of new oncology drugs. Cohort of trials from 25 drugs granted first-time Food and Drug Administration (FDA) approval for oncology indications in 2000 to 2005 and published in 1996 to 2008.
Data137 RCTs of oncology drugs (placebo or active control).
ComparisonsIndustry sponsor and other funding.
OutcomesStudy results, study conclusions, methodological quality (blinding) and concordance between study results and conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE and The Cochrane Library.
Control for bias?YesLogistic regression adjusted for confounders.

Rattinger 2009

MethodsTo examine the association between research funding source, study design characteristics aimed at reducing bias, and other factors with the results and conclusions of RCTs of thiazolidinediones compared to other oral hypoglycemic agents (search 1996 to 2006).
Data61 RCTs of thiazolidinediones (active or placebo control).
ComparisonsTest drug company, other drug company, all others and not declared funding.
OutcomesStudy results, study conclusions, methodological quality (sequence generation and allocation concealment, blinding and follow-up) and concordance between study results and conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE, The Cochrane Library, references and reviews.
Control for bias?YesIntended multivariate analysis, but due to few associations only univariate performed.

Ridker 2006

MethodsTo determine in contemporary randomized cardiovascular trials the association between funding source and the likelihood of reporting positive findings. Cardiovascular RCTs published in JAMA, Lancet, and the New England Journal of Medicine in 2000 to 2005.
Data349 RCTs (mixed interventions and comparators).
ComparisonsFor profit, mixed and not for profit funding.
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesSample of journals identified via MEDLINE.
Control for bias?NoNo control for bias.

Rios 2008

MethodsTo assess the reporting quality of RCTs in general endocrinology and to identify predictors for better reporting quality. RCTs published in the Journal of Clinical Endocrinology and Metabolism, Clinical Endocrinology, and the European Journal of Endocrinology in 2005 or 2006.
Data89 endocrinology drug RCTs (various comparators).
ComparisonsIndustry, mixed, non-industry and not stated funding.
OutcomesMethodological quality (allocation concealment and blinding).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?YesTwo assessors included studies.
Comprehensive search?YesHandsearch of journals.
Control for bias?YesControlled for confounders using multivariate analysis.

Rochon 1994

MethodsTo study the relation between reported drug performance in published trials and support of the trials by the manufacturer of the drug under evaluation. All non-steroidal anti-inflammatory (NSAID) RCTs from September 1987 to May 1990.
Data56 NSAID RCTs (placebo and head-to-head comparisons).
ComparisonsManufacturer associated only.
OutcomesStudy results (efficacy and harms), study conclusions (efficacy and harms) and methodological quality (Chalmers' scale, 0-100 points).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE searched.
Control for bias?NoControl for bias seems unlikely to have been done.

Tulikangas 2006

MethodsTo determine if there is a significant difference in outcomes of clinical trials funded by industry or not of antimuscarinic medications used to treat overactive bladder symptoms and detrusor overactivity. RCTs from 1980 to 2002.
Data24 RCTs of antimuscarinic drugs (various comparators).
ComparisonsIndustry funded and public funded.
OutcomesStudy results.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE and references.
Control for bias?NoNo control for bias.

Tungaraza 2007

MethodsTo compare drug trials reported in three major psychiatric journals to investigate whether treatments are more likely to report favorable outcomes when they are funded by the pharmaceutical industry. Studies published in the British Journal of Psychiatry, American Journal of Psychiatry and Archives of General Psychiatry from 2000 to 2004.
Data198 psychiatric drug trials (various designs and comparators).
ComparisonsIndustry sponsored, industry authored and independent.
OutcomesStudy conclusions.
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesHandsearch of journals.
Control for bias?NoNo control for bias.

Vlad 2007

  1. a

    RCT: Randomised controlled trial

MethodsTo identify factors that explain heterogeneity in trials of glucosamine. RCTs of glucosamine from 1980 to 2006.
Data15 RCTs of glucosamine versus placebo for osteoarthritis.
ComparisonsIndustry funding, industry participation, industry author and independent.
OutcomesStudy results, effect size and methodological quality (allocation concealment and Jadad score, 0-5 point scale).
Notes 
Risk of bias
ItemAuthors' judgementDescription
Adequate selection criteria?YesObjective criteria used.
Adequate study inclusion process?UnclearDoes not describe number of assessors.
Comprehensive search?YesMEDLINE, The Cochrane Library, conference abstracts, references and reviews.
Control for bias?YesExploration of heterogeneity.

Characteristics of excluded studies [ordered by study ID]

StudyReason for exclusion
Chowers 2009No relevant outcomes
Conen 2008No relevant outcomes
Cunningham 2007No separate drug or device data
Friedman 2004Conflicts of interest, not funding
Glick 2006No relevant outcomes
Hall 2007No relevant outcomes
Hill 2007No relevant outcomes (not methodological quality, but reporting quality)
Jagsi 2009No separate drug or device data
Khan 2008No separate drug or device data
Kjaergard 1999No separate drug or device data
Krzyzanowska 2003No relevant outcomes
Kulier 2004No quantitative data
Kulkarni 2007No relevant outcomes
Lai 2006No separate drug or device data
Leopold 2003No separate drug or device data
Leucht 2009aNo relevant outcomes
Leucht 2009bNo relevant outcomes
McLennan 2008No relevant outcomes
Montori 2005No relevant outcomes
Nkansah 2009Calcium supplementation, not a drug
Okike 2007Conflicts of interest, not funding
Okike 2008No relevant outcomes
Procyshyn 2004No relevant data for non-industry studies
Roach 2008No separate drug or device data
Sanossian 2006No relevant outcomes
Shah 2005No separate drug or device data
Thomas 2008No relevant outcomes (not methodological quality, but reporting quality)
Watanabe 2010No relevant outcomes
Yao 2007No separate drug or device data
Yaphe 2001No separate drug or device data

Ancillary