Disclosure of conflicts of interest in biomedical research is receiving increased attention. The authors sought to define current disclosure policies and how they relate to disclosure statements provided by authors in major oncology journals.
The authors identified all oncology journals listed in the Thomson Institute for Scientific Information and sought their policies on conflict-of-interest disclosure. For a subset of journals with an Impact Factor >2.0, they catalogued the number and type of articles and the details of the published disclosures in all papers from the 2 most recent issues.
Disclosure policies were provided by 112 of 131 journals (85%); 99 (88%) of these requested that authors disclose conflicts of interest (mean Impact Factor for these journals: 4.6), whereas the remaining 13 (12%) did not (mean Impact Factor: 2.9). Ninety-three journals (94%) required financial disclosure, and 42 (42%) also sought nonfinancial disclosures. For a subset of 52 higher-impact journals (Impact Factor >2.0), we reviewed 1734 articles and identified published disclosures in 51 journals (98%). Many of these journals (31 of 51, 61%) included some disclosure statement in >90% of their articles. Among 27 journals that published editorials/commentaries, only 14 (52%) included disclosures with such articles. There was no publication of any nonfinancial conflicts of interest in any article reviewed.
Researchers, clinicians, and the public are increasingly concerned about the potential for investigators' financial relationships to create conflicts of interest in the conduct or reporting of biomedical research.1 There is growing evidence about the possible effects of these conflicts; sources of financial support have been shown to have an impact on the design2 and outcomes3 of trials and on the reporting of results4 in many medical fields,5-8 including oncology.9 Because eliminating potential conflicts of interest is essentially impossible and may even be harmful,10 nearly all biomedical journals manage this issue by requiring authors to disclose funding for their work, as well as other relevant relationships that they, their families, or their institutions might hold when an article is submitted for publication.11 Certain disclosure policies have been shown to affect the presentation of published articles.12
In some cases, researchers have not been forthcoming about potential financial conflicts. For example, a trial about use of computerized tomography to help detect lung cancer in smokers listed substantial funding from a nonprofit foundation that, 2 years later, was revealed to have close financial ties to a major cigarette manufacturer.13 Medical journals, for their part, have been criticized for publishing articles without fully disclosing or adequately accounting for financial relationships of authors, such as in supplements wholly sponsored by pharmaceutical manufacturers.14, 15
As a result, many journals have re-examined their conflict of interest disclosure practices in recent years. In 2009,16 the International Committee of Medical Journal Editors promulgated new standards (revised in 2010),17 including a uniform disclosure template that requires authors to disclose direct support for the research, personal financial relationships including honoraria, stock options and patents, and other interests authors might have related to their work. Despite these advances, biomedical journals still have a range of disclosure practices. Recent studies of journals with high Impact Factors identified substantial variation in disclosure requirements and in how conflict of interest is defined in cardiology, across medical specialties, and in pediatrics.18-20
In the field of cancer research, financial relationships and other conflicts of interest of biomedical researchers have been particularly contentious.21, 22 Previous studies have identified frequent reports of conflicts of interest in papers published in high-impact journals.23, 24 To more comprehensively define the current status of journal disclosure practices in this field, we collected information related to conflict of interest disclosure from all oncology journals. We also examined the disclosures accompanying papers published in these journals to compare the nature of requested information with information that is provided alongside published research.
MATERIALS AND METHODS
Sources of Data
We identified all oncology-related medical journals using the Thomson Institute for Scientific Information's Web of Science. From September to December 2009, we searched each journal's official website and copied the entire author submission guidelines. Any separate documents related to manuscript submission were downloaded and compiled into a single complete record. Journals that did not provide author submission information on their website were contacted by email (on 2 separate occasions, as necessary) to request all guidelines or forms from an editor.
We then selected a subset of these oncology journals for more detailed investigation of the published disclosures through an analysis of published articles. We limited this analysis to all journals from our original sample with an Impact Factor >2.0 based on the 2008 Impact Factor published by the Thomson Institute for Scientific Information Journal Citation Reports. Impact Factor is a ratio of the number of citations in the current year to items published by a journal in the previous 2 years (the numerator) and the number of substantive articles published by that journal in the last 2 years (the denominator).25 We used Impact Factor because this measure has been found to be the strongest predictor of an article's subsequent dissemination.26 In addition, articles from journals with higher Impact Factors have been preferentially cited in government-developed practice guidelines, which are intended to influence physician behavior.27 Finally, focusing on the highest-impact journals allowed us to more easily obtain full access to these journals through pre-existing institutional library electronic or print subscriptions. In this analysis, we acquired each article from the 2 most recent issues preceding the date that we obtained their disclosure policy (date range: June-December 2009) and copied any published author disclosures from all research articles, reviews, and commentaries/editorials.
Design of the instrument for recording information about disclosure policies was informed by review of the literature and a preliminary review of the disclosure requests from 5% of the journals in this sample. The information collected fell into 3 categories. First, we noted characteristics of the journal, including its country of origin, Impact Factor, and whether it was a member of a publishing group (eg, Nature, Elsevier). We identified whether the journal had a disclosure requirement statement, the word length and reading level of that disclosure statement, and the existence (if any) of a preprinted form. Reading level was assessed using the Flesch-Kincaid Grade Level, which is a reliable and valid tool that uses word and sentence length to determine the school-grade reading level of text.28 Reading level is a proxy for the understandability of the forms and the ease with which they can be completed. These factors may influence the substance of disclosures, even for highly educated individuals. Next, we extracted general features of the disclosure policy, including its definition and scope, specific mention of funding source or ghostwriting, the particulars of publication of the disclosure, and penalties for nondisclosure. Finally, we identified disclosure descriptors, including types of financial and/or nonfinancial relationships and mention of particular dollar figures or date ranges.
The instrument for recording the content of published disclosures followed a similar structure. We catalogued the number and type of articles (eg, original research, editorial) and the location of the printed disclosure (if any). If a disclosure was included with the article, we noted which disclosure descriptors were included. So as not to overemphasize journals that published a relatively large number of articles per issue, we set our unit of analysis as the journal (rather than the article). If any article within the 2 issues we analyzed included an author disclosure, that journal was registered as publishing this descriptor.
Each journal's author disclosure requirements and content of printed disclosures were evaluated separately by 2 authors (A.S.K., A.S., J.L.L., or N.K.C.). For journals' definitions of the basic scope of disclosure, which were a primary outcome (Fig. 1) and required subjective assessment on the part of the coders, we assessed the level of interobserver agreement by calculating a kappa statistic, which is a measure of the level of agreement beyond chance. For classifying a journal's disclosure policy as including information that could lead to author embarrassment, the kappa was 0.84. For classifying a journal's disclosure policy as including information that could lead to the results being biased, the kappa was 0.75. For classifying a journal's disclosure policy as including information relating to outside benefits earned by the authors, the kappa was 0.64. These results indicate good-to-excellent interobserver agreement29 and are comparable to results from previously published descriptive research in this field.24 All differences were resolved by consensus. We used Excel 2003 (Microsoft, Redmond, Wash) for descriptive analyses.
We identified 131 oncology-related journals and were able to obtain information about disclosure policies from 112 (85%) of them. Ninety-nine (88%) of these journals requested author disclosures, whereas the remaining 13 either did not request disclosures from authors or did not have disclosure policies (see Table 1). Journals in this latter category had a relatively low mean Impact Factor (2.9) and were most likely to be based in Europe (54%) and have a thematic focus on oncologic basic science (38%). The data extracted from the sample of 99 journals that requested author disclosures formed the sample for our analysis of disclosure policies.
Table 1. Characteristics of Oncology Journals Studied
Journals Requiring Disclosure, n=99
Journals With No Policies, n=13
Abbreviations: ICMJE, International Committee of Medical Journal Editors; IQR, interquartile range; SD, standard deviation.
Total N = 131. No information on disclosure policies could be obtained for 19 journals (15%).
Does not include 3 journals that referred to ICMJE policies but were not registered with the ICMJE.
Among journals that sought author disclosures, 56 (57%) had individualized disclosure policies, 40 (40%) had disclosure policies derived from a larger family of journals (eg, Elsevier, Nature group, American Association for Cancer Research), and 3 (3%) had a general statement that referred to the current International Committee of Medical Journal Editors guidelines. Thirty-seven (37%) had separate forms related to disclosure, whereas the remainder (63%) requested that authors include a free-form disclosure statement along with their manuscript. The combined disclosure statements and forms averaged 360 words and were at a 13.8 grade reading level.
A minority of journals took additional steps to describe the disclosure process in more depth. Thirty-four (34%) provided a descriptive rationale for why their policy was important. Eleven journals (11%) provided comments that disclosures were routine or that they would not automatically lead to a manuscript's rejection; 13 (13%) specified penalties for failure to disclose a relevant relationship. Twelve journals (12%) promised that disclosures would be kept confidential, and 8 (8%) noted that disclosures would not be shared with peer reviewers. Finally, 39 (39%) described some features of how a disclosure might be published in conjunction with an accepted manuscript.
Definitions of disclosure
Sixty-seven journals (68%) made some reference to the types of outside relationships that would qualify for disclosure, although only 50 (51%) explicitly defined the term conflict of interest. Concerning relationships were most commonly described in 1 of 3 nonmutually exclusive categories: those that could bias conclusions or influence the integrity or objectivity of the author (51, 52%), those with companies that could stand to gain from the outcome of the trial (24, 24%), and those that could embarrass the author (13, 13%) (Fig. 1). A minority of journals (13, 13%) recommended erring on the side of caution when deciding what to disclose. Nearly half of the journals explicitly limited disclosures to the author (48, 49%), whereas smaller numbers also requested potential relationships from authors' families (12, 12%), institutions (10, 10%), or both (3, 3%). The remaining 26 journals (26%) did not specify the relevant parties for disclosure purposes.
Specifics of financial versus nonfinancial disclosures
Ninety-three journals (94%) specifically requested financial disclosures, and although no journal requested the particular amount, 11 (11%) provided guidance for what should be considered a relevant monetary disclosure. For example, 1 journal suggested that a $10,000 financial interest was relevant for employment, stocks, and patents, $20,000 for research funding, and $5000 for honoraria, manuscript fees, or other gifts. A minority of journals, 42 (42%), also sought nonfinancial disclosures. Journals provided a number of different examples of the types of financial and nonfinancial disclosures authors should disclose (Fig. 2). Twenty-nine (29%) specified the time frame for lapsed relationships (median 3 years, interquartile range, 3-3), and even fewer (11, 11%) requested information about planned or future relationships. Finally, a small number of journals (27, 27%) asked whether any authors were paid to write the manuscript without participating in the research (ghostwriting), and a similar number (30, 30%) asked for details about the role of the funding source.
Characteristics of Published Disclosures
From the sample of 99 journals that required disclosure, 68 qualified for further review (Impact Factor >2.0), and we obtained full copies of 2 issues of 52 (76%) of the journals in this group. We reviewed 1734 articles from these journals (an average of 17 original articles per issue, including an average of 0.7 editorials/commentaries). We identified printed disclosures in 51 of the 52 (98%) journals. The dominant location (40, 78%) for appearance of the published disclosure was after the text of the article. The remainder placed the disclosures on the first page (4, 8%), both on the first page and after the text (5, 10%), or after the references (2, 4%). Among these 51 journals, nearly two-thirds (31 of 51, 61%) included some disclosure statement (including reports of financial interests or a printed statement confirming lack of conflicts) in >90% of the articles they published. The remainder included disclosure statements in smaller fractions of their articles (Fig. 3); 1 journal included a disclosure statement of any kind in only a single article in the issues we reviewed. Rates of disclosure statements differed between reports of research and editorials/commentaries. Among the 27 journals that published editorials/commentaries, only 14 (52%) included disclosures along with that content.
Although 51 journals included at least 1 disclosure relating to research grants, the prevalence of other published descriptors was much smaller (Fig. 2). Nonfinancial conflicts of interest were commonly requested; “personal ties” were specifically mentioned by nearly 40% of journals in the original sample. However, we found no publication of nonfinancial conflicts of interest in any of the issues that we obtained. Finally, 6 journals (12%) included a disclosure that referred to a time period when the conflict existed, and 3 (6%) included a disclosure that referred to the role of the funding source.
This analysis reveals a wide range of disclosure policies and practices in peer-reviewed oncology journals. Most but not all of the journals we examined required some disclosure of potential conflicts of interest, related standards and definitions varied considerably. Although both financial and nonfinancial disclosures were required, in the 104 issues we reviewed, no nonfinancial disclosures were reported.
The observation that nearly every journal in this field now includes a disclosure policy reflects a substantial evolution from past practices30 and suggests that editors have taken seriously the importance of transparency of investigators' conflicts of interest, allowing readers to make their own decisions about how such conflicts might influence the design or reporting of biomedical research. However, the practice is still not universal; 13 journals (12%) had no such policy, including 10 with Impact Factors >2.0. It is possible that the actual percentage is even higher, because 19 journals did not respond at all to our requests for a description of their disclosure policies. As we have found in previous studies in other fields, publication of disclosures for editorials and commentaries lags far behind those for reports of research.31, 32 Such overviews may represent precisely the kind of articles that are most vulnerable to perceived or unconscious bias.
In addition to identifying important disclosure gaps in certain journals and types of articles, we also found little reporting of select types of conflicting relationships. First, it is unclear how much time must elapse until past relationships no longer may reasonably be expected to influence behavior, although a small minority of journals provided details about the time period covered by the disclosure. Planned or future relationships may be even more relevant than those in the past, and such disclosures, although occasionally requested, were found associated with only 1 of the 1734 articles in our sample. Second, few journals required authors to disclose potential conflicts from family members or institutions, and only 6 disclosure statements in the 1734 articles we reviewed (in 4 different journals) contained any such information. These types of relationships have received comparatively less attention in the medical literature, yet examples from the government33 and media34 suggest that the relationships of one's spouse, for example, are viewed as legitimate conflicts of interest. Transparency on these conflicts may be difficult to enforce, but we believe the current landscape of widely varying requirements from journals in this field only serves to diminish the perceived importance of these conflicts to authors and readers.
We also documented wide variations in how journals define conflict of interest; relatively few described their disclosure policies in sufficient depth to help guide authors' disclosures. Many journals disclosed only rudimentary information about revealed financial relationships (ie, Dr. A is a consultant for drug company XYZ), without providing any details or measure of scale, so that a $500 honorarium cannot be distinguished from a large annual retainer. Studies of pay-for-performance incentives for physicians document that financial incentives constituting <2% of a physician's annual income can lead to sustained changes in behavior.35 Although it may be impossible to define a precise level at which financial conflicts become relevant, even arbitrarily defined thresholds could help readers evaluate the importance of disclosed relationships.
These results support the need for a more uniform depiction of conflict of interest rules in peer-reviewed journals, as well as more predictable policies concerning which of these conflicts will be published. Some organizations, such as the International Committee of Medical Journal Editors, have offered standardized disclosure statements. Any standardization efforts, however, should allow journals flexibility to adapt the disclosure policies to their particular scientific area. For example, the types of disclosures relevant to a journal focusing on basic science might be different from those relevant to a clinical science or health policy journal. There remain numerous advantages to a broader coordination of disclosure policies. One obvious benefit would be to ensure the dissemination of minimum standards that might lead to extinguishing certain practices, such as ghostwriting, that all scientific journals should reject.36 Such an effort could also help medical journals share effective practices and achieve consensus on definitions of key terms or parameters of disclosure, including time limits on past work or minimum financial thresholds. One way to develop minimum standards would be to identify which policies optimize transparency, for example by comparing authors' disclosures across different journals, or by comparing authors' disclosures to new government-mandated databases of financial relationships.37 Journals could also consider organizing a central repository that could more efficiently oversee author disclosures.
We believe that standardized disclosure practices will assist readers of the medical literature in critically reading published articles, and will encourage responsible reporting by authors and journals. To accomplish the former goal, disclosed conflicts should be more consistently reported in a way that promotes clarity and assists readers. For example, we found that some journals provided disclosures extending for over a page in length that appeared to include all financial relationships, including those that may not be relevant to the topic of the article. This approach to disclosure reporting could be counterproductive if readers are not given any guidance about the importance of different disclosures. Ideally, disclosure should advance critical evaluation of study methods and conclusions, but should not lead readers to discount well-designed studies, regardless of sponsorship or authors' relationships. Small randomized studies have found that readers of articles with disclosures of conflicts show increased skepticism about articles with disclosed conflicts, as compared with articles without disclosures,38, 39 but trials have not identified how variations in the presentation or content of the disclosure affect interpretation of the published work.40
Another potential value of enhanced disclosure could be its effect on authors. Researchers and clinicians who are aware of basic standards for transparency may be more likely to provide evidence-based, rather than partisan, recommendations if they know their disclosures will subject them to higher scrutiny by readers. Some behavioral-psychological research has suggested that disclosure may enhance biases by giving subjects moral license to provide more extreme opinions.41 The solution is not to reduce transparency,42 but both to enhance transparency and to encourage accuracy through peer review, effective editing by publishers, critical review by readers, and confirmatory testing where necessary. The Journal of the American Medical Association, for example, has required all industry-funded studies to undergo independent statistical testing before submission.43
Our analysis has certain limitations. It is an in-depth evaluation of 1 field of medicine, potentially limiting its generalizability. It is also based on publicly available information regarding journals' policies and the disclosures printed with published articles. It is unclear whether the absence of published disclosures in certain categories (eg, for nonfinancial conflicts) reflects a lack of policy on this point by journals, a failure of disclosure by authors, or decisions by editors not to publish such disclosures. Finally, our study is a cross-sectional look at a single moment in a rapidly changing field; the articles we reviewed may have been the product of work several years prior when journals' policies were different.
Despite these limitations, our review documents substantial variation in what is requested and what is ultimately printed in oncology journals regarding authors' potential financial and nonfinancial conflicts of interest. We found frequent disclosure of stock ownership, consulting arrangements, and receipt of honoraria in oncology journal articles. Relationships with for-profit organizations have sparked intense debate within the profession and in the lay media, although such relationships can help generate and fund useful innovation. Because such relationships are common among clinicians and researchers publishing articles in oncology journals, a more coherent approach may be required to promote transparency in this rapidly evolving area.
A.S.K.'s work is supported by a career development award from the Agency for Healthcare Research and Quality (K08HS18465-01) and a Robert Wood Johnson Foundation Investigator Award in Health Policy Research.