Received from The Social Policy Research Institute (LSF), Skokie, Ill; and Hebrew University-Hadassah School of Community Medicine and Public Health, Unit of Occupational and Environmental Medicine, Injury Prevention Center (LSF, EDR), Jerusalem, Israel.
Relationship Between Conflicts of Interest and Research Results
Article first published online: 28 JAN 2004
Journal of General Internal Medicine
Volume 19, Issue 1, pages 51–56, January 2004
How to Cite
Friedman, L. S. and Richter, E. D. (2004), Relationship Between Conflicts of Interest and Research Results. Journal of General Internal Medicine, 19: 51–56. doi: 10.1111/j.1525-1497.2004.30617.x
- Issue published online: 28 JAN 2004
- Article first published online: 28 JAN 2004
- conflict of interest;
- publication bias;
- biomedical ethics;
CONTEXT: To date, research regarding the influence of conflicts of interest on the presentation of findings by researchers has been limited.
OBJECTIVE: To evaluate the sources of funding for published manuscripts, and association between reported findings and conflicts of interest.
METHODS: Data from both print and electronic issues of The New England Journal of Medicine (NEJM) and The Journal of the American Medical Association (JAMA) were analyzed for sources of funding, areas of investigation, conflict of interest (COI), and presentation of results. We reviewed all original manuscripts published during the year 2001 within NEJM (N = 193) and JAMA (N = 205). We use 3 definitions for COI in this paper: a broadly defined criterion, the criterion used by The International Council of Medical Journal Editors (ICMJE), and a criterion defined by the authors.
RESULTS: Depending on the COI criteria used, 16.6% to 32.6% of manuscripts had 1 or more author with COI. Based on ICMJE criterion, 38.7% of studies investigating drug treatments had authors with COI. We observed a strong association between those studies whose authors had COI and reported positive findings (P < .001). When controlling for sample size, study design, and country of primary authors, we observed a strong association between positive results and COI (ICMJE definition) among all treatment studies (adjusted odds ratio [OR], 2.35; 95% confidence interval [CI], 1.08 to 5.09) and drug studies alone (OR, 2.64; 95% CI, 1.09 to 6.39).
CONCLUSION: COI is widespread among the authors of published manuscripts and these authors are more likely to present positive findings.
The influence of commercial interests on the principal players in the peer review process—researchers, reviewers, and editors—is an important and sensitive issue facing biomedical research. There is a need to comprehensively evaluate the degree of influence, direct and indirect, of private corporations on researchers.
Studies have found the presence1–4 and absence5 of an association between funding from pharmaceutical companies and presentation of positive findings. However, none are readily available which have addressed this association among research funded by all health care industry manufacturers or within the broader framework of a conflicts of interest (COI) definition, which includes not only direct funding in the form of grants but also other types of personal financial associations and interests such as consultancy, employment, stock ownership, patent licensing, and honoraria. In addition, our review of the literature shows no study to date which has addressed the likelihood of publishing negative findings among authors with COI, which answers an entirely different question.
In this study we evaluate the sources of funding for published manuscripts. We also examine whether the reported relationship between positive findings and financial associations are sustained when using a COI definition and including nonpharmaceutical manufacturers in our analysis. Furthermore, we investigate the likelihood of publishing negative findings given COI.
We selected the top 2 general medical journals based on their year 2000 impact factor as ranked by ISI Journal Citation Reports.6 Data from both the print and electronic 2001 issues of The New England Journal of Medicine (NEJM) and The Journal of the American Medical Association (JAMA)7,8 were analyzed for trends in sources of funding, areas of investigation, COI, and presentation of results. All monetary descriptions are in U.S. dollars.
In NEJM, we analyze only manuscripts defined by the journal as “Original Reports.” In JAMA, we analyze 190 “Original Contribution” articles, and an additional 15 in the following subsection headings: 3 “Caring for critically ill patients,” 3 “Clinical cardiology,” 2 “Clinical evaluation,” 4 “Clinical investigation,” 2 “Preliminary communications,” and 1 “Toward optimal laboratory use.” Editorials, reviews, commentaries, case reports, and brief reports from both journals were excluded.
We use author descriptions to classify study design, number of subjects per study, and funding source. In cases where author/s did not define study design, we use the definitions in Abramson for categorization.9
Both journals request authors to disclose financial relationships with companies whose product they review in the manuscript. Sponsor and type of financial support were in most cases disclosed by authors. The authors of 33 studies from both journals combined did not disclose financial relationships. All 33 studies were categorized as studies without financial associations. Seven studies in which the authors only provide the name of a financial sponsor, but not a description of the financial support, were categorized as “grant/funding.”
We were unable to find broad consensus for any single definition of COI. We therefore use 3 definitions for COI in this paper: 1) a broadly defined criterion, 2) the criterion used by The International Council of Medical Journal Editors (ICMJE), and 3) a criterion defined by the authors.
The broadly defined COI criterion is defined as all financial relationships with companies whose products the researchers are evaluating in the manuscript, except for studies only supported by free drugs and equipment.
Second, we use the COI criterion set forth by the ICMJE.10 This “narrowly defined” criterion refers to those financial relationships specifically cited as the most severe examples of conflict of interest that include consultancy, employment, stock ownership, patent licensing, and honoraria. This criterion excludes financial relationships based on grants, both general and specific funding, awards, fellowships, free drugs or equipment, and authors serving as speakers or on an advisory board.
Third, we use an internal definition of COI. Neither of the above criteria address the commercial components of the studies. A study must meet each of the following 4 criteria: 1a) one or more authors have financial associations with a private corporation in the form of grants, unspecified funding, consultancy, employment, stock ownership, or honoraria; and/or b) have a personal financial interest in the study because of a patent license in which an author is eligible to receive royalties or from personal investments. Free drugs and equipment, awards, fellowships, and serving on advisory boards or as speakers do not constitute COI in this definition; 2) drug/treatment/product reviewed by the author/s is manufactured by the funding corporation, or is in the same retail class11 as a drug manufactured by a sponsoring competitor; 3) product(s) reviewed by author(s) must have current or near future commercial potential (i.e., sold for profit); and 4) presentation of main findings support commercial product, negate competitor's product, advocate cost benefit, and/or show product has a potential commercial value (demand, size, and growth).
To assess the association between COI and reported study findings, we classified the presentation of the results as follows: positive, mixed, negative, or other.
Positive results include studies that show a statistically significant (P < .05) clinical benefit from a treatment or absence of suspected side effects (P > .05), achieve statistically valid equivalence comparable to commonly used therapies, or support their product by observing side effects in a competitive product or insignificant association with intended outcome from use of competitor's product (P > .05; N = 4 studies for latter). Mixed results include studies noting both clinical benefits from a treatment and presence of significant adverse effects (both P < .05). Negative results include studies that report the absence of clinical benefits (P > .05) and/or evidence for numerous adverse effects (P < .05). “Other” category includes studies that are observational or cross-sectional emphasizing frequency and distribution rather than comparison between groups; trends in medical services and product usage; drug discontinuation protocol; or studies whose significance is yet unclear because it is a preliminary or pilot study.
We use SAS for Windows (version 8.0; SAS Institute, Inc., Cary, NC) for the statistical analysis. We use χ2 tests for analysis of categorical variables. We fit a multiple logistic regression model including potential predictors to estimate adjusted odds ratios (OR) for reported positive/negative results and COI. We use both ICMJE and author-defined criteria for COI when fitting our model. Adjusted odds ratios for all treatment-related studies and drug-related studies, respectively, were calculated. The model included the following variables: sample size, study design, and country of origin of primary authors. To evaluate the association between positive results and COI, the categories used in Table 3 were aggregated (mixed, negative, and other). The same procedure was conducted to evaluate the association between negative results and COI (positive, mixed, and other were aggregated). A two-sided P value less than .05 was considered statistically significant.
|All||Positive n (%),*||Mixed n (%),†||Negative n (%),‡||Other n (%),§||P Value″|
|Drug treatment studies with COI||60||51 (85.0)||7 (1.7)||1 (1.7)||1 (1.7)|
|Drug treatment studies without COI||59||24 (40.7)||9 (15.3)||21 (35.6)||5 (8.5)||< .001|
|All treatment studies with COI||73||61 (83.6)||10 (13.7)||1 (1.4)||1 (1.4)|
|All treatment studies without COI||101||53 (52.5)||14 (13.9)||28 (27.7)||6 (5.9)||< .001|
|Drug treatment studies with COI||46||36 (78.3)||9 (19.6)||1 (2.2)||0 (0.0)|
|Drug treatment studies without COI||73||39 (53.4)||7 (9.6)||21 (28.8)||6 (8.2)||< .001|
|All treatment studies with COI||57||45 (78.9)||10 (17.5)||2 (3.5)||0 (0.0)|
|All treatment studies without COI||117||69 (59.0)||14 (12.0)||27 (23.1)||7 (6.0)||< .01|
Table 1 shows the general characteristics of original manuscripts published in NEJM and JAMA during 2001.
|Circulation (paid subscriptions only)*||193,785||249,532|
|Number of Original Manuscripts published in 2001||193||205|
|Prevalence/cross-sectional, % (n)||2.6 (5)||15.1 (31)|
|Case-control, % (n)||16.1 (31)||4.9 (10)|
|Cohort, % (n)||24.4 (47)||32.7 (67)|
|Clinical trial, % (n)||48.2 (93)||24.9 (51)|
|Evaluation/validation, % (n)||3.1 (6)||16.6 (34)|
|Other,†% (n)||5.7 (11)||5.9 (12)|
|Mean number of subjects per study‡,§ (range)||8763 (2 to 679,942)||47,266 (2 to 1,900,000)|
|With ≤ 100 subjects, %||26.4||12.0|
|With ≥ 1,000 subjects, %||31.6||54.9|
|Funding Source§, (n)|
|Area of Investigation|
|Health care—providers, physicians, and patients, % (n)||0 (0)||16.6 (34)|
|Health promotion and primary prevention, % (n)||1.6 (3)||6.3 (13)|
|Risk assessment, % (n)||25.4 (49)||30.2 (62)|
|Screening (diagnostic and prognostic), % (n)||9.9 (19)||10.7 (22)|
|Treatment (primary through tertiary), % (n)||59.6 (115)||27.3 (56)|
|Other, % (n)||3.6 (7)||8.8 (18)|
In 2001, NEJM and JAMA, respectively, published 193 and 205 original manuscripts. A total of 72.6% of the original manuscripts in NEJM were clinical trials or cohort studies compared to the 57.6% in JAMA. JAMA contained a larger number of cross-sectional and evaluation/validation studies (difference, 15%; 95% confidence interval (CI), 3%, 27%).
Large samples were frequently used in the manuscripts of both journals. Fifty-five percent of the manuscripts in JAMA had sample sizes ≥ 1,000.
A total of 81.9% (N = 158/193) of original manuscripts in NEJM and 87.8% (N = 180/205) in JAMA received funding from government and/or private corporations. Private corporations alone funded 38.3% and 34.6% of the research articles in NEJM and JAMA, respectively. The 5 companies most frequently reported as study sponsors were GlaxoSmithKline, Aventis, Merck, Pfizer, and Hoffman-LaRoche.
Original manuscripts published in JAMA covered a broader scope of investigations than those published in NEJM, which primarily focused on risk assessment and treatment (85% of total; P < .001). A total of 16.6% of the original manuscripts in JAMA discussed health care issues of providers, physicians, and patients, whereas no original manuscripts in this area of investigation were published by NEJM.
Conflict of Interest
Table 2 notes the number of original manuscripts in which 1 or more authors reported financial associations with private corporations or had personal financial interests in the study product (patents, stock).
|Articles (N)||% of Total (N, 193)||Articles (N)||% of Total (N= 205)|
|Type of financial interest or association as reported by authors||76||39.4||76||37.1|
|Patent licensed to author||4||2.1||5||2.4|
|Served as speakers||4||2.1||13||6.3|
|Articles in Which One or More Authors Had a Conflict of Interest|
|Broadly defined criterion*||63||32.6||53||25.9|
Based on our internally defined conflict of interest criterion, the authors of 27.5% of the original manuscripts in NEJM had potential COI compared to 20.0% in JAMA. When using the ICMJE criterion (including only consultancy, employment, stock ownership, patent licensing, and honoraria), the authors of 22.3% of the original manuscripts in NEJM had potential COI compared to 16.6% of the manuscripts in JAMA (Table 2).
Table 3 shows the distribution of study outcomes for manuscripts with and without COI. There exists a strong association between those studies whose authors had a COI and reported positive findings (P < .001). Based on the ICMJE criterion, we observed that 38.7% (46 out of 119) of studies investigating drug treatments had authors with COI compared to 20.0% (11 out of 55) of studies investigating nonpharmaceutical therapies.
Based on the ICMJE criterion, when controlling for sample size, study design, and country of primary authors, we observed a strong association between positive results and COI among all treatment studies (adjusted odds ratio [OR], 2.35; 95% CI, 1.08 to 5.09) and drug studies alone (adjusted OR, 2.64; 95% CI, 1.09 to 6.39). Using the author-defined COI criterion, we observed a strong association between positive results and COI among all treatment studies (adjusted OR, 4.07; 95% CI, 1.90 to 8.72) and drug studies alone (adjusted OR, 7.32; 95% CI, 2.87 to 18.71).
However, the strength of the association increased when comparing reported negative results and COI (ICMJE definition) for all treatment studies (adjusted OR, 0.107; 95% CI, 0.02 to 0.49) and drug studies only (adjusted OR, 0.05; 95% CI, 0.01 to 0.43). Based on our internal definition of COI, the relationship between reported negative results and COI for all treatment studies was OR = 0.03 (adjusted; 95% CI, 0.004 to 0.251) and for drug studies only was OR = 0.02 (adjusted; 95% CI, 0.001 to 0.189). The odds are extremely small that negative results would be published by authors with COI.
Conflicts of Interest
Private corporations funded approximately 1 out of every 3 original manuscripts published in the largest 2 general medicine journals in the United States. Depending on the COI criterion, prevalence of COI by 1 or more authors varies between 19.4% and 29.2% of all original manuscripts published in both journals combined.
When in 1999 NEJM reported 19 drug review articles with apparent COI, they hoped to reduce the number the following year.12 NEJM editors have contended that past failures to contain conflict of interest have been the result of “poor communication and coordination” among its editorial staff.12 Recent easing of NEJM's COI rules for editorial and review articles may reflect the growing difficulty in finding articles from authors without ties to private industry.
NEJM cites the view13 that conflict of interest is a condition, not a behavior, in which the circumstances and not the outcome determine the presence of COI. Our data suggest that the condition is pervasive. Furthermore, based on the ICMJE criterion, authors with COI were 10 to 20 times less likely to present negative findings than those without COI. The relationship was strongest among studies investigating drug treatments.
Based on a review of the literature, our study is the first to report an association between financial associations and reported findings using COI criterion. Past studies have focused on direct funding, which neglects other important forms of personal financial associations and interests which are addressed in a COI criterion such as consultancy, employment, stock ownership, patent licensing, and honoraria. In addition, our study shows that the relationship between funding and reported findings persists when including nonpharmaceutical companies in the analysis. While past studies have focused on positive findings, this study also addresses the likelihood of publishing negative findings among authors with COI.
We attempt to integrate commercial aspects of the product evaluated in our COI criterion. Because one of the criteria is that the main findings support the study product, our definition for COI should isolate only studies with positive or mixed findings. In the one case in which the findings were negative, the primary conclusion of the authors from both the abstract and discussion state that the negative effects of the study drug may be the result of an operational bias. We felt that the manner in which the findings were presented to a degree support the product. Despite the bias that may arise from the use of our criterion, the direction of the results based on our definition are replicated when using the ICMJE definition.
Because of the limitations of our data, we are unable to determine the reasons for the observed association between COI and reported findings. One could surmise that drug companies are selective and only want to invest in treatments proven to produce positive results and that early clinical trials filter out the most promising treatments, which could explain the small number of studies funded by private corporations presenting negative findings. But we find 21 studies without corporate funding reporting negative findings regarding on-the-market drugs compared to only 1 study funded directly or indirectly by corporations (Table 3). Furthermore, the concern that Phase I clinical trials will bias results toward positive findings may not be valid. Because Phase I trials focus on the drug's pharmacokinetics and maximum tolerated dose in a small sample of healthy individuals, it is unlikely that subclinical and rare side effects would be revealed at this phase of investigation, particularly side effects more likely to develop in sick individuals.
The question arises as to whether an investigator with a conflict of interest may be more inclined to present findings in order to gain favor with the sponsor or achieve any other extraneous objective—e.g., to “spin.” The issue of spinning findings goes beyond the lower likelihood to criticize the safety or efficacy of a treatment1,14 or the withholding of data on adverse reactions.15
It appears that some companies selectively sponsor projects in which their drug is not evaluated but the findings are likely to support their commercial interests. We noted 5 studies in which the researchers focus on the shortcomings of a competitor's product or observe side effects resulting from the use of a widely prescribed therapy which can be treated by the sponsor's product. None of these studies mention the sponsor's drug in the analysis. Most COI policies fail to address these types of studies. These studies may be a good example of exploiting market pressures as a means of doing quality control on drugs, even though the motivations for funding such studies probably reflect commercial interests rather than therapeutic concerns.
The observation that negative findings are less commonly reported among studies funded by private corporations raises troublesome ethical questions. Researchers appear to be failing to promote both the benefits and negative side effects of commercial products they review or simply failing to submit negative studies for publication because they are viewed as uninteresting.16 On the other hand, editors are not proactively examining the possibility of bias from author relationships with private corporations.17
Furthermore, there is no system to effectively regulate and oversee researchers and journals. The current federal financial disclosure regulations do not require institutions to comprehensively collect, review, and disclose information on all significant financial interests in research irrespective of the source.18
At the academic level, it is unclear whether COI review committees,18 as proposed by the Association of American Medical Colleges, would be effective in managing COI since they lack the ability to mandate the formation of review bodies and enforcement of disciplinary measures. In addition, the independence of academic bodies themselves is questionable considering academic faculties in the United States receive approximately $1.5 billion annually in research money from private industry.19 This is a possible explanation for academe's strong resistance to external government regulation of their funding.20
Further research is required which clearly defines the parameters of COI (criterion, prevalence, strength of bias). In addition, greater detail is necessary about the types of remuneration received by authors in order to evaluate whether $10,000 is an appropriate federally defined cutoff for mandatory disclosure of significant financial relationships related to sponsored research and should consider other nonfinancial types of gratuity currently ignored.
Private health care companies heavily invest in “independent” researchers. Those researchers with COI are more likely to present positive findings. Pharmaceutical companies spent approximately $23 billion on clinical research in 2001 as compared with $18 billion from the National Institute of Health.21 Physicians often begin receiving pharmaceutical gifts and remuneration as early as the first year of medical school.22 These investments establish long-term relationships with the “middle-man” (i.e., clinical researchers) in order to have access to study populations23 and capitalize upon the notion of consensual validity these “objective” independent researchers have among consumers.
Though remuneration does not necessarily result in unethical behavior, it can be a strong catalyst for it.18 The need for independent researchers has long been understood, yet a large proportion of research continues to be conducted by those with COI. Today's system of oversight appears to be ineffective in monitoring COI among researchers. External regulation of data integrity and financial associations should be discussed as an avenue to monitor COI.
- 6ISI Journal Citation Report 2000. Available at: http://www.isinet.com/isi/. Accessed February 5, 2002.
- 7N Engl J Med. 2001: 344–345: 1–1978 and 1–1882.
- 8J Am Med Assoc. 2001;285–286: 1–3129 and 1–3119.
- 9Survey Methods in Community Medicine, 4th edn. Churchill Livingstone;1990:ch. 2..
- 10International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. Updated October 2001.
- 11Drug Monitor, IMS Health Global Services. 2002. Available at: http://www.imshealth.com. Accessed July 17, 2002.
- 18Task Force on Financial Conflicts of Interest in Clinical Research. Protecting Subjects, Preserving Trust, Promoting Progress Policy and Guidelines for the Oversight of Individual Financial Interests in Human Subjects Research. Association of American Medical Colleges; 2001.
- 21Temptation to fudge scientific findings. The Virginian-Pilot and The Ledger-Star, Norfolk, Va. November 12, 2000.