ACADEMIC EMERGENCY MEDICINE 2011; 18:977–980 © 2011 by the Society for Academic Emergency Medicine
Objectives: This study was a review of the scientific abstracts presented at a national conference for the required conflict of interest (COI) disclosure both before the meeting and during presentation.
Methods: All presenters were given specific instructions regarding COI reporting at the time of abstract acceptance. All poster presentations were required to have a COI statement. Three physicians using standardized data abstraction forms reviewed abstracts accepted for poster presentation at the 2010 annual meeting of the Society for Academic Emergency Medicine (SAEM). Posters were reviewed for the presence of a required COI disclosure statement, and these results were compared to the mandatory continuing medical education (CME) disclosure form that was sent by the presenters to the SAEM central office before the meeting.
Results: There were 412 posters accepted for presentation at the 2010 SAEM annual meeting. The reviewers observed 382 (93%) of the total posters for the conference. Sixty-nine abstracts (18%) reported a COI. Only 26 (38%) of these were actually reported to the SAEM office on the CME disclosure form before the meeting; the remaining 62% were found on the poster alone. COI that were reported on the CME disclosure form were found on the poster 46% of the time. The remaining posters without a COI actually displayed the mandatory disclosure statement only 14% of the time.
Conclusions: This review of presentations at a national meeting found a lack of compliance with printed guidelines for COI disclosure during scientific presentation. Efforts to increase uniformity and clarity may result in increased compliance.
The role of a scientific presentation at a clinical meeting is to make available to a forum of colleagues interesting or relevant advances that can affect the health and welfare of patients. An open process of communication allows investigators to integrate new information with existing clinical practices, stimulates discussion, and generates new areas of inquiry. Critical to this process is the knowledge of the financial motivations and potential sources of conflict that lie behind any presented research. All professional medical associations require that investigators disclose potential conflict of interest (COI) before and during any presentation or discussion of research findings.
Our goal was to evaluate the scientific abstracts accepted for poster presentation at the 2010 annual meeting of the Society for Academic Emergency Medicine (SAEM) for the presence of the required COI disclosure statement and the correlation of that statement with the continuing medical education (CME) disclosure form submitted to the SAEM central office.
This was an observational cohort study. This research met criteria for exemption from review by the institutional review board of the New York Methodist Hospital. The primary goal of this study was to see if the reporting of financial COI for original research posters as reported before the annual meeting (on CME forms) matched what was actually presented at the conference. The secondary goal of this investigation was to measure compliance with mandatory reporting guidelines published before the meeting requiring a disclosure statement to be present on the poster for original research abstracts regardless of whether a COI existed or not.
Study Setting and Population
The study was conducted at the 2010 SAEM annual meeting in Phoenix, Arizona. Original research abstracts presented in printed poster format were evaluated for the presence of the COI disclosure statement required by SAEM.
The construct of the analysis was to evaluate the compliance of investigators with rules published by SAEM for the disclosure of potential sources of conflict in the presentation of the results of original research presentations. Presenters were electronically mailed specific instructions prior to the meeting about the intent and meaning of a COI disclosure, the importance of a priori reporting to the central SAEM office, as well as appropriate disclosure during all activities at the annual meeting. These instructions included a CME disclosure form for the meeting as a CME event sponsored by the University of California, Irvine. This disclosure form, shown online in Data Supplement S1, required the lead author to disclose potential conflicts (including but not limited to funding, employer relationships, or honoraria of any amount). Additionally, investigators who were selected for poster presentation received an instruction sheet educating presenters on the correct format to be used for the design of their poster. This instruction sheet clearly stated that all scientific presentations must include disclosure of any funding source. Both the CME disclosure form and the requirement of a COI disclosure statement were listed as mandatory on the acceptance letter sent to authors. The nature and location of the disclosure statement (even if there were no conflicts of interest) was further clarified in the online and print versions of the instructions for poster presentations.
Three experienced investigators used standardized data abstraction forms to review all of the available poster presentations 1 hour before scheduled presentation time. Posters were reviewed for the presence of the mandatory disclosure statement regarding conflicts of interest anywhere on the poster, as well as the actual disclosure of a COI. Each poster presenting original research was scored as “No Disclosure Statement” made, stated “No Conflicts,” or “COI stated” on the poster. Each poster was evaluated by two independent reviewers.
The CME disclosure forms mailed to the SAEM central office before the meeting were reviewed for reported COI and paired with the specific poster presentations reviewed. The reviewers at the annual meeting were unaware of the content of the CME disclosure forms at the time of the abstract review.
Data for all measured variables are reported as percentages. Interrater reliability between the three reviewers was assessed using Cohen’s kappa statistic. All analyses were conducted using R: a language and environment for statistical computing (R Foundation for Statistical Computing, 2009, Vienna, Austria).
There were 609 original research abstracts accepted for presentation at the 2010 SAEM annual meeting; of those, 412 were poster presentations. The three reviewers observed 382 (93%) of the total posters for the conference (no reviewers were available for the final day of the conference). Each poster was reviewed by two independent reviewers. In the final analysis, if one reviewer observed that a COI was reported or a COI statement was present, that observation was used.
Of these, the reviewers identified 69 abstracts (18%) with a reported COI. Only 26 (38%) of these were actually reported to the SAEM office on the CME disclosure form before the meeting; the remaining 43 (62%) were printed on the poster, but not reported to the SAEM office. When there was a financial COI reported to the SAEM office on the CME form, that information was placed on the poster for the abstract only 46% of the time.
The posters that were classified as not having a COI actually displayed a disclosure statement on only 53 (14%) posters. The remaining 86% were merely presumed to have no conflict since no statement to the contrary was present at the SAEM office or printed on the poster.
The poster presentations were the focus of this review; however, all three reviewers also observed the plenary session abstracts. This included the five best original research abstracts of the meeting presented in oral format to the entire Society. A financial COI was reported on the CME disclosure form for four of the five plenary sessions abstracts, but only two of those four reported the conflicts during their talks.
Data from all three investigators were compared, and agreement of 0.70 (95% confidence interval = 0.69 to 0.71) was calculated. The major source of discrepancy between reviewers occurred when material items (and not money) were provided by an external agency. It was decided after review (and after calculating agreement) that this would be coded as “COI stated” because material goods were provided to support the research activity.
The need for transparency in the conduct of clinical research has long been recognized as crucial to the peer review process and the dissemination of novel results to the scientific community. Despite this, reporting of financial ties that can confound a trial is a crucial first step in the presentation of research findings that can easily be overlooked. Recent cases of high-profile and very public research misconduct linked with ongoing investigations by the Office of the Inspector General (OIG) have kept the issue of research integrity and the potential for commercial influences in the forefront of the public eye.1–5 Every professional medical organization has standards for the reporting of COI during the presentation of research findings, whether in print, by poster, or in oral format.
No medical organization, to our knowledge, reports a systematic or structured plan to ensure that these rules for the disclosure of COI are followed, nor are consequences for a failure to report these COI easy to identify. Only three specialties have published the results of an audit of the COI disclosures made at their annual scientific meetings: otolaryngology, oncology, and orthopedics.6–12 Orthopedics has been by far the best studied, due in part to the nature of the specialty and in large part to the release of data from a recent fraud settlement by the OIG.5,13 Orthopedic surgery is heavily dependent on prosthetic devices that are often invented or improved by practicing physicians in conjunction with industry, and it is common for the physician to hold the patent to the device and have a vested interest in its use. In a 2007 antikickback settlement with the OIG, the five manufacturers holding a 95% market share of prosthetic hip devices agreed to disclose all payments made to physicians.5,13 Using these data, a study published in 2009 found that presenters, committee members, or board members of the 2008 annual meeting of the American Academy of Orthopaedic Surgeons only disclosed these payments 71% of the time.12 The most common reasons these physicians gave for not reporting were a belief that the topic of the presentation was unrelated (49%) and misunderstanding the instructions (13%). Independent review of significance of the COI found that the reporting rate when the COI was directly related to the topic of presentation was 79%; indirectly related, 50%; and not related, 49%.12 While addressing only one narrow area of financial COI in a medical specialty fraught with complex financial arrangements between physicians and industry, this study illustrates the limitations of self-reporting and highlights the need for uniform COI reporting. The establishment of uniform reporting standards would likely reduce error, and mandating enforcement of more universal reporting is needed to avoid misunderstandings about whether or not conflicts are relevant to a subject presentation.
The inability of an individual medical organization to effectively audit COI disclosures has led to a reliance on self-reporting at the societal and the individual level. Our review of the COI disclosures provided at the societal and individual level during presentation at the 2010 SAEM annual meeting would suggest that self-reporting without regulation is ineffective. The rules governing the reporting of COI to the society for CME events have existed for years, yet fewer than half of the COI that were self-reported at the meeting in our sample actually made it to the CME disclosure form sent to the SAEM office before the meeting, and fewer than half of those COI reported on the disclosure form were actually reported at the meeting as required. Whether by design or simple neglect, the reporting of financial conflicts is not being disseminated to the audience that needs to know that information.
One reason that investigators may not disclose a COI is a failure to recognize that a relationship actually represents a potential problem in the conduct or reporting of research results. Our review did not attempt to grade the seriousness or complexity of the financial relationship, since the purpose of the disclosure is to allow the members of the audience to make their own assessments. The instructions provided to presenters did have some ambiguity in that the CME disclosure form asked for the “company” that was associated with the conflict, although the instructions for presenters specifically stated that “all sources of funding must be disclosed.” Adding to the confusion, all of these forms and requirements were newly introduced or modified for the 2010 annual meeting. The culture of the annual meeting has simply not placed an emphasis on the reporting of financial conflicts, and inconsistent policies coupled with a lack of accountability have resulted in poor compliance and limited understanding. This could have led investigators to feel that it was not necessary to report some types of financial relationships on the CME disclosure form. In fact, only 26 of the 382 (7%) posters reviewed actually reported conflicts both on the CME disclosure form and at the meeting. The largest discrepancy between reporting on the CME disclosure form and reporting on the poster had to do with funding that was received from public sources; typically this was not reported on the CME disclosure form. It is certainly possible that the belief that public sources of funding do not create a COI coupled with the language on the CME disclosure form led some presenters to not disclose the conflict to the SAEM office before the meeting. Of the three societies publishing results of a COI audit at their annual meetings (Academy of Otolaryngology-Head and Neck Surgery Foundation, American Academy of Orthopaedic Surgeons, and the American Society of Clinical Oncology), all required the reporting of external funding (public, private, or foundation) and treated them as equivalent for reporting purposes. It makes sense to consider that the external funding provided by public sources can provide just as much of a conflict as that provided by a commercial source when successful competition for public grants directly affects institutional overhead, promotion and tenure, salary, and personal recognition. The Office of Research Integrity, which investigates allegations of research misconduct involving public health service funds, has averaged 150 new allegations of academic misconduct involving public health service funds per year for the past decade, including a very-high-profile case of data fabrication by a clinician at Harvard University.4,14 Another investigator (and institution) avoided public disclosure of a commercial source of funding by accepting the funds routed through a private foundation established for that purpose, until that link was published in the lay press.15
The obvious limitation of this audit is that the financial COI were self-reported. It should be evident in reading through these results that the process for reporting a COI both to the central SAEM office and in print on the poster is far from understood by the presenters and is so inconsistent as to render the disclosure process largely meaningless. Our observed rate of 82% of abstracts without any COI seems low when compared to other specialties and must be tempered with the fact that 86% of those abstracts did not actually make the required disclosure statement on the poster. The posters were not evaluated for conflicts related to the specific presentation as the instructions for presentation requested all conflicts be presented. As mentioned, this differed from the CME form and may have resulted in underreporting on the CME form.
The current system for reporting and relaying conflicts of interest in original research is inconsistent and suffers from a lack of transparency and oversight. The process could likely be improved by adapting the information required on the continuing medical education form to be consistent with the instructions provided to presenters and by requiring reporting of all financial conflict of interest, regardless of relatedness to the scientific presentation. Further education of investigators who submit abstracts of original research to the Society for Academic Emergency Medicine annual meeting should be undertaken in this regards. A system of monitoring and oversight may also be required to establish effective and meaningful conflict of interest disclosure during presentation.