Incorporating Evidence-based Medicine into Resident Education: A CORD Survey of Faculty and Resident Expectations

Authors

  • Christopher R. Carpenter MD, MSc,

    1. From the Departments of Emergency Medicine: Washington University in St. Louis, School of Medicine (CRC), St. Louis, MO; Lehigh Valley Hospital-Muhlenberg (BGK), Bethlehem, PA; Albert Einstein Medical Center (MC), Philadelphia, PA; The George Washington University (RL), Washington, DC; Indiana University School of Medicine (LGW), Indianapolis, IN; and Eastern Virginia Medical School (CSG), Norfolk, VA.
    Search for more papers by this author
  • Bryan G. Kane MD,

    1. From the Departments of Emergency Medicine: Washington University in St. Louis, School of Medicine (CRC), St. Louis, MO; Lehigh Valley Hospital-Muhlenberg (BGK), Bethlehem, PA; Albert Einstein Medical Center (MC), Philadelphia, PA; The George Washington University (RL), Washington, DC; Indiana University School of Medicine (LGW), Indianapolis, IN; and Eastern Virginia Medical School (CSG), Norfolk, VA.
    Search for more papers by this author
  • Merle Carter MD,

    1. From the Departments of Emergency Medicine: Washington University in St. Louis, School of Medicine (CRC), St. Louis, MO; Lehigh Valley Hospital-Muhlenberg (BGK), Bethlehem, PA; Albert Einstein Medical Center (MC), Philadelphia, PA; The George Washington University (RL), Washington, DC; Indiana University School of Medicine (LGW), Indianapolis, IN; and Eastern Virginia Medical School (CSG), Norfolk, VA.
    Search for more papers by this author
  • Raymond Lucas MD,

    1. From the Departments of Emergency Medicine: Washington University in St. Louis, School of Medicine (CRC), St. Louis, MO; Lehigh Valley Hospital-Muhlenberg (BGK), Bethlehem, PA; Albert Einstein Medical Center (MC), Philadelphia, PA; The George Washington University (RL), Washington, DC; Indiana University School of Medicine (LGW), Indianapolis, IN; and Eastern Virginia Medical School (CSG), Norfolk, VA.
    Search for more papers by this author
  • Lee G. Wilbur MD,

    1. From the Departments of Emergency Medicine: Washington University in St. Louis, School of Medicine (CRC), St. Louis, MO; Lehigh Valley Hospital-Muhlenberg (BGK), Bethlehem, PA; Albert Einstein Medical Center (MC), Philadelphia, PA; The George Washington University (RL), Washington, DC; Indiana University School of Medicine (LGW), Indianapolis, IN; and Eastern Virginia Medical School (CSG), Norfolk, VA.
    Search for more papers by this author
  • Charles S. Graffeo MD

    1. From the Departments of Emergency Medicine: Washington University in St. Louis, School of Medicine (CRC), St. Louis, MO; Lehigh Valley Hospital-Muhlenberg (BGK), Bethlehem, PA; Albert Einstein Medical Center (MC), Philadelphia, PA; The George Washington University (RL), Washington, DC; Indiana University School of Medicine (LGW), Indianapolis, IN; and Eastern Virginia Medical School (CSG), Norfolk, VA.
    Search for more papers by this author

  • Presented at the Council of Emergency Medicine Residency Directors Academic Assembly, Orlando, FL, March 4, 2010.

  • Disclosures: This publication was made possible by Grant Number UL1 RR024992 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NCRR or NIH. Dr. Carpenter is a faculty member for Best Evidence in Emergency Medicine and was supported by the Washington University Goldfarb Patient Safety award.

  • Supervising Editor: John Burton, MD.

Address for correspondence and reprints: Christopher R. Carpenter, MD, MSc; e-mail: carpenterc@wustl.edu.

Abstract

Background:  The Accreditation Council for Graduate Medical Education (ACGME) invokes evidence-based medicine (EBM) principles through the practice-based learning core competency. The authors hypothesized that among a representative sample of emergency medicine (EM) residency programs, a wide variability in EBM resident training priorities, faculty expertise expectations, and curricula exists.

Objectives:  The primary objective was to obtain descriptive data regarding EBM practices and expectations from EM physician educators. Our secondary objective was to assess differences in EBM educational priorities among journal club directors compared with non–journal club directors.

Methods:  A 19-question survey was developed by a group of recognized EBM curriculum innovators and then disseminated to Council of Emergency Medicine Residency Directors (CORD) conference participants, assessing their opinions regarding essential EBM skill sets and EBM curricular expectations for residents and faculty at their home institutions. The survey instrument also identified the degree of interest respondents had in receiving a free monthly EBM journal club curriculum.

Results:  A total of 157 individuals registered for the conference, and 98 completed the survey. Seventy-seven (77% of respondents) were either residency program directors or assistant/associate program directors. The majority of participants were from university-based programs and in practice at least 5 years. Respondents reported the ability to identify flawed research (45%), apply research findings to patient care (43%), and comprehend research methodology (33%) as the most important resident skill sets. The majority of respondents reported no formal journal club or EBM curricula (75%) and do not utilize structured critical appraisal instruments (71%) when reviewing the literature. While journal club directors believed that resident learners’ most important EBM skill is to identify secondary peer-reviewed resources, non–journal club directors identified residents’ ability to distinguish significantly flawed research as the key skill to develop. Interest in receiving a free monthly EBM journal club curriculum was widely accepted (89%).

Conclusions:  Attaining EBM proficiency is an expected outcome of graduate medical education (GME) training, although the specific domains of anticipated expertise differ between faculty and residents. Few respondents currently use a formalized curriculum to guide the development of EBM skill sets. There appears to be a high level of interest in obtaining EBM journal club educational content in a structured format. Measuring the effects of providing journal club curriculum content in conjunction with other EBM interventions may warrant further investigation.

ACADEMIC EMERGENCY MEDICINE 2010; 17:S54–S61 © 2010 by the Society for Academic Emergency Medicine

Evidence-based medicine (EBM), perhaps more aptly named “research-enhanced practice,” is optimally the overlap of clinical expertise, best available evidence, and a patient’s unique values, all of which guide bedside decision-making in a dynamic fashion.1,2 The use of EBM tools can provide a structured, efficient approach for clinicians based on two guiding principles. First, not all evidence is created equal. In other words, a hierarchy of evidence exists, with weaker evidence manifest by increasing potential for bias. The second principle is that evidence alone is never enough, and the absence of evidence is not synonymous with evidence of absence.1 Evidence does not make decisions; people do. Authoritarian management decisions should not exclude individual patient preferences.3

Evidence-based medicine principles are increasingly important for emergency medicine (EM) residents to learn. Seasoned clinical expertise is invaluable, but is sometimes insufficient to ensure high-quality health care.4 Half of patients already fail to receive guideline-directed management.5 Traditionally, continuing medical education has been the primary mode of postresidency knowledge acquisition, but didactic conferences inconsistently improve knowledge acquisition or yield practice change.6,7 Additionally, industry can and has manipulated clinician behavior through the use of opinion leaders and marketing strategies that commonly misrepresent research findings.8–12 The Accreditation Council for Graduate Medical Education (ACGME) and the Council of Emergency Medicine Residency Directors (CORD) define the following elements for the core competency of practice-based learning:

  • 1Locate, appraise, and use scientific evidence related to the patient’s health problems and the larger population from which they are drawn;
  • 2Apply knowledge of study design and statistical methods to critically appraise the medical literature;
  • 3Use information technology to enhance personal education and improve patient care.13

At a minimum, EM residency graduates need to have acquired a healthy skepticism for innovations as the science of knowledge translation advances the frontiers of health care.14,15 EBM is one framework often referenced for developing these competencies in residents, but trainees are being provided inadequate or inconsistent learning experiences. A recent survey of 65 EM program directors found that while 80% report EBM within their curriculum, only 22% provide more than 5 hours of exposure annually.16 Barriers to EBM dissemination in residency programs include inadequate time and trained personnel, incomplete awareness of EBM resources, insufficient funding, and faculty apathy.16 Although anecdotal reports of successful postgraduate EBM models have been described,17 skeptics question the data to support these models and EBM.18,19

Our primary objective was to obtain descriptive data on current EBM practices and anticipated outcomes from a representative group of EM educators attending the CORD annual academic assembly. Our secondary objective was to conduct a stratified analysis of journal club directors compared with non–journal club directors, with the hypothesis that the majority of EBM knowledge acquisition in EM residencies is self-contained within the journal club educational model. To address our study hypothesis, we sought to analyze sustainable, locally successful models of EM graduate medical education (GME) EBM curricula from diverse EM residency settings for educational elements that could be applied in other residency settings to overcome common barriers.

Methods

Study Design and Population

In March 2010, CORD offered seven nonoverlapping workshops in best practices in EM residency training over 3 days as part of the annual academic assembly. More than 100 program directors and academic EM faculty attended the workshop on “Incorporating Evidence Based Medicine Into Your Residency.” In preparation for the didactic workshop, two authors of this article (CRC, CSG) solicited the CORD membership for examples of sustained, well-established components of GME EBM curricula. Among respondents, a panel of interested EBM leaders was formed in September 2009 to delineate the objectives for the didactic session and this article. This survey study was approved by the Eastern Virginia Medical School Institutional Review Board.

Survey Content and Administration

A survey instrument to explore current attitudes towards and actual teaching of EBM was developed via iterative feedback with a qualitative research expert. The final survey instrument was distributed three weeks before the CORD meeting via e-mail to all best practices track registrants (n = 157) with a follow-up e-mail 5 days before the meeting for all nonrespondents. The survey was also handed out in paper format at the CORD meeting, and attendees were asked to hand in their responses at the end of the didactic session. Those who had completed the e-mail survey were asked not to turn in a paper copy as well.

The investigators planned a priori to conduct a subset analysis of journal club directors compared with other respondents to identify significant differences representing opportunities or barriers to promoting EBM curricular initiatives in GME. This analysis was planned based on the hypothesis that journal club represents the vehicle by which EBM is taught in EM residencies.

All survey responses were entered into an Excel database (Microsoft Corp., Redmond, WA) on a double password–protected computer maintained behind two locked doors. Prior to releasing the data to investigators, the responses were stripped of all identifiers by administrative personnel not involved in the study design or manuscript preparation. All duplicate responses were eliminated prior to data entry.

Data Analysis

Univariate data analysis was performed using SPSS (version 16.0, SPSS, Inc., Chicago IL) by one author (CRC) employing chi-square analysis for dichotomous proportional data and factorial logistic regression analysis when more than two categorical responses were possible.

Results

The survey was completed by 98 participants, including 34 via e-mail and 64 at the CORD annual academic assembly. There were no significant differences between e-mail and written respondents in experience or job title. Two respondents (a resident and non-EM physician) were eliminated from this analysis to provide a more homogeneous sampling of CORD faculty attendees. The results for the remaining 96 respondents are described in Table 1. The majority of respondents (91%) were beyond 5 years postresidency and practicing in a university-based EM training program (57%). Also, the majority of respondents serve as either the program director or the associate/assistant program director (77%). While 25% personally administer journal club, almost half (48%) attend journal club on a relatively infrequent (<60%) basis. The majority of respondents do not use either a formal critical appraisal instrument (71%) or an established EBM curriculum for journal club (75%). Most (78%) do not use any extramural sources for journal club, such as ACP Journal Club, Best Evidence in Emergency Medicine, EM Abstracts, or Annals of Emergency Medicine Journal Club.20–24 Although most respondents are not currently using available EBM resources, the large majority (89%) expressed interest in a free monthly EBM curriculum to use as their journal club if one were available.

Table 1. 
Overall Survey Responses (N = 96)
 Respondents (%*)
  1. EBM = evidence-based medicine; GME = graduate medical education.

  2. *Proportions may not add up to 100% due to rounding and exclusion of nonrespondents.

  3. †Examples of Cochrane library, EM Abstracts, and Best Evidence in Emergency Medicine provided to survey participants.

Years postresidency
 1–49 (9)
 5–934 (35)
 10–1938 (40)
 >2015 (16)
Practice setting
 University-based EM residency55 (57)
 Community-based EM residency35 (36)
 Affiliate hospital with EM resident rotator1 (1)
 Hospital with no EM residents2 (2)
 Other3 (3)
Academic position
 Chair or vice chair9 (9)
 Program director29 (30)
 Associate or assistant program director45 (47)
 Research director3 (3)
 Site director1 (1)
 Academic faculty, clinical researcher1 (1)
 Academic faculty, educational researcher3 (3)
 Academic faculty, other4 (4)
 Clinically affiliated faculty1 (1)
Personally teach EBM principles60 (63)
Personally administer journal club24 (25)
Personal annual journal club attendance
 Once or twice (1%–20%)11 (11)
 A few times (21%–40%)14 (15)
 Frequently (41%–60%)21 (22)
 Almost always (61%–80%)28 (29)
 Always (81%–100%)22 (23)
Use structured critical appraisal instrument(Users’ Guide)28 (29)
Use extramural source for journal club21 (22)
Have predetermined journal club curriculum24 (25)
Single most important EBM skill—all EM physicians
 Formulate question7 (7)
 Search strategy7 (7)
 Understand research methodology32 (33)
 Determine ideal study statistical approach1 (1)
 Differentiate statistical and clinical significance6 (6)
 Apply research results to individual patients41 (43)
Most important outcome of GME EBM curricula—residents
 Become expert critical appraisers26 (27)
 Differentiate minimally and significantly flawed studies43 (45)
 Ability to find reliable secondary peer reviewed material27 (28)
Most important outcome of GME EBM curricula—faculty
 Become expert critical appraisers53 (55)
 Differentiate minimally and significantly flawed studies32 (33)
 Ability to find reliable secondary peer reviewed material9 (9)
EM residents must be able to critically appraise research
 Strongly disagree7 (7)
 Disagree12 (13)
 Neutral9 (9)
 Agree43 (45)
 Strongly agree24 (25)
EM faculty must be able to critically appraise research
 Strongly disagree7 (7)
 Disagree6 (6)
 Neutral11 (12)
 Agree36 (37)
 Strongly agree33 (34)
EM resident critical appraisal proficiency
 None0
 Minimal18 (19)
 Moderate52 (54)
 High22 (23)
 Very high4 (4)
EM faculty critical appraisal proficiency
 None0
 Minimal4 (4)
 Moderate30 (31)
 High43 (45)
 Very high17 (18)
EM residents should know how to use secondary sources†
 Strongly disagree11 (12)
 Disagree2 (2)
 Neutral3 (3)
 Agree30 (31)
 Strongly agree50 (52)
EM faculty should know how to use secondary sources†
 Strongly disagree7 (7)
 Disagree0
 Neutral3 (3)
 Agree29 (30)
 Strongly agree56 (58)
Interested in free monthly EBM curriculum85 (89)

Survey respondents identified the application of research findings to individual patients (43%) and understanding research methodology (33%) as the quintessential EBM skills for resident and faculty emergency physicians. Expected GME EBM curricula outcomes for resident physicians and faculty physicians differed. Resident physicians were expected to finish their training with the ability to differentiate minimally biased research from significantly flawed studies (45%), while faculty were to develop sufficient capabilities to become expert critical appraisers (55%). Most respondents felt that residents (71%) and faculty (74%) need to possess sufficient critical appraisal expertise to make appropriate real-time patient care decisions during clinical shifts. Most felt that residents (83%) and faculty (88%) should learn how to identify and access secondary peer-reviewed sources relevant to their patient’s care following an effective GME EBM program.

The analysis comparing journal club directors to non–journal club directors is summarized in Table 2. Journal club directors were not more likely to teach EBM principles, use structured critical appraisal forms, or have an established journal club curriculum. More non–journal club directors favored the ability to distinguish minimally and significantly flawed research (25% of journal club directors vs. 52% of non–journal club directors; p = 0.05). Journal club directors instead favored the ability to find and use secondary peer-reviewed sources (46% vs. 21%; p = 0.05). Most physicians felt that finding and using secondary sources (e.g., ACP Journal Club, Best Evidence in Emergency Medicine) should be part of the skill set for residents (88% in both groups) and faculty (80% among journal club directors, 92% among non–journal club directors). Both journal club directors and nondirectors identified the ability to apply research findings to individual patients as the single most important EBM skill for all EM physicians (42% journal club directors and 40% of non–journal club directors).

Table 2. 
Survey Responses Stratified by Journal Club Director (N = 91)*
 Journal Club Directors (%)Non–Journal Club Directors (%)
  1. *Five respondents did not answer this question.

  2. †Examples of Cochrane library, EM Abstracts, and Best Evidence in Emergency Medicine provided to survey participants.

  3. EBM = evidence based medicine; GME = graduate medical education; PD = program director.

Academic position
 Program director, associate or assistant PD20 (83)52 (78)
 Non-PD4 (17)15 (22)
Personally teach EBM principles18 (75)40 (60)
Personal annual journal club attendance
 > 40%21 (88)45 (67)
Use structured critical appraisal instrument (Users’ Guide)8 (33)19 (29)
Use extramural source for journal club7 (30)13 (20)
Have predetermined journal club curriculum6 (26)17 (27)
Single most important EBM skill—all EM physicians
 Formulate question2 (9)5 (8)
 Search strategy2 (9)5 (8)
 Understand research methodology8 (35)23 (35)
 Determine ideal study statistical approach01 (1)
 Differentiate statistical & clinical significance1 (4)5 (8)
 Apply research results to individual patients10 (42)27 (40)
Most important outcome of GME EBM curricula—residents
 Become expert critical appraisers7 (29)18 (27)
 Differentiate minimally and significantly flawed studies6 (25)35 (52)
 Ability to find reliable secondary peer-reviewed material11 (46)14 (21)
Most important outcome of GME EBM curricula—faculty
 Become expert critical appraisers11 (46)38 (58)
 Differentiate minimally and significantly flawed studies9 (38)23 (35)
 Ability to find reliable secondary peer-reviewed material4 (17)4 (6)
EM residents must be able to critically appraise research
 Strongly disagree or disagree4 (17)15 (22)
EM faculty must be able to critically appraise research
 Strongly disagree or disagree4 (17)9 (13)
EM resident critical appraisal proficiency
 None00
 Minimal3 (13)13 (19)
 Moderate13 (54)38 (57)
 High7 (29)13 (19)
 Very high1 (4)3 (4)
EM faculty critical appraisal proficiency
 None00
 Minimal2 (8)2 (3)
 Moderate6 (25)23 (34)
 High12 (50)28 (42)
 Very high4 (17)12 (18)
EM residents should know how to use secondary sources†
 Strongly disagree or disagree3 (12)8 (12)
EM faculty should know how to use secondary sources†
 Strongly disagree or disagree5 (20)5 (8)
Interested in free monthly EBM curriculum23 (96)59 (88)

Discussion

This is the largest survey to ascertain EM GME EBM curricular practice and expectations to date. Our sampling of EM educators suggests that understanding research methodology, finding reliable sources of secondary peer-reviewed content, and applying research findings within the context of patient care are important skills for all emergency physicians. These findings are consistent with the ACGME requirements for developing “systems-based practices” that encourage appropriate and efficient use of medical, social, and technological systems, with the ultimate goal of improving patient care.25,26 Despite this recognition among our respondents, a minority have an established EBM curriculum, routinely use user guides to the medical literature, or take advantage of extramural peer-reviewed resources. These findings appear to support our hypothesis that wide variability in EBM resident training, faculty expertise, and curricula exists.

For more than a century, journal club has been the de facto vehicle to develop critical appraisal expertise during residency.27 Traditionally, journal club has focused on research data appraisal without expanding to include the EBM competencies of defining a clinical question before finding the evidence, communicating complicated scientific interpretations to patients to inform health care decision-making, or knowledge translation.28 Another limitation of the traditional journal club approach is that negative studies are frequently not published.29–31 Furthermore, the effect of journal club on resident EBM skill uptake is inconclusive.32–35 Finally, literature linking new evidence acquisition to practice change and the ultimate goal of improved patient outcomes as the result of journal club or EBM curricula is lacking.35–37

One means of moving the traditional journal club toward one embedded with EBM is to use PICO (PICO = patient, intervention, comparison, outcome) questions. Focusing on resident generated or “real world” clinical problems as a source for choosing articles helps add relevance.38 Additionally, journal club websites and databases allow clinicians timely access to previously appraised topics during patient care. Multiple programs archive their journal club–derived critically appraised topics for public access on the Internet.39,40 One of these programs has published the results of its journal club infrastructure, instructional instruments, and final product reviews in the EM literature.17,41–46 These archives serve the function of internally developed secondary sources, and the websites include links to free online statistical calculators, external secondary peer-reviewed synopses, and other institutions’ EBM curricula.47 This practice is entirely consistent with the viewpoints of EM program directors in our survey who felt that the ability to both analyze research articles and access secondary peer-reviewed sources are important skills. An additional benefit of adopting use of these websites and databases may be to cultivate an EBM culture and warrants further investigation.

Another method of creating a culture of research-enhanced practice within a residency program may include providing incoming residents with introductory EBM lectures during orientation, including topics such as the formulation of relevant clinical questions in the form of PICOs, efficient Web-based search strategies for primary and secondary peer-reviewed sources, and an introduction to key EBM resources such as the Users’ Guide to the Medical Literature.38,48 An additional means to embed EBM into a didactic program is to replace core content lectures with a resident-formulated, faculty-mentored literature review and critical appraisal prior to presenting to their peers. This approach has already been described in the literature.49,50

Previously identified barriers to EBM dissemination in residency include inadequate instructional time and trained personnel, incomplete awareness of EBM resources, insufficient funding, and faculty apathy.16 In our opinion, fundamental elements for success include departmental chair support, shared faculty expertise, and a local EBM champion.51–53 Integrating EBM into other areas of the didactic curriculum and into clinical practice may offer a more holistic EBM teaching approach that extends beyond journal club as the sole pedagogical approach. It also increases the available instructional time previously noted to be lacking. Evidence-based workshops that teach question formation or effective search strategies can be efficient ways to teach fundamental skills, such as forming PICO questions and learning literature search strategies. These workshops can be augmented by enlisting expertise from reference librarians or guest faculty.54

It has been suggested that residents should generate clinically relevant PICO questions based on their patients. The residents should then proceed to document their search strategy, the time required for their search, and their conclusions for each question based on the search.55 This process has been successfully employed in at least one EM residency program. The anticipated benefit of such an experience should be the resident, in real time, connecting EBM to patient care. One immediate objective of an EBM curriculum could be teaching residents and faculty how to find and utilize high-quality secondary peer-reviewed information sources that few are currently using.54,56 One means to document such utilization is to track online electronic reference resource use by end-users. Proprietary software packages (e.g., Horizon Business Insight, McKesson Corp., San Francisco, CA) make it possible to determine utilization practices for resident and faculty regarding digital library holdings.57 Institutional library services departments are potential partners in this effort, as the documentation of database use by clinicians may assist in justifying their budgetary process. This documentation may have the potential added benefit of serving as ACGME “Phase 3” outcome data. One author’s institution has instituted this online resource tracking (BGK).

Emergency medicine residency programs can quantify learners’ EBM knowledge acquisition using the Fresno test, a validated 12-question measure of EBM skills.58 The results could be used to inform curricular development. When any program is both designing its residency EBM curriculum and delineating its faculty development plan, our data suggest that it is important to recognize that expectations for EBM expertise differ for faculty and residents. Whereas residents are expected to attain the ability to identify significantly flawed studies, faculty should master critical appraisal of the medical literature. These differences should direct local administrative and educational leaders to provide sufficient opportunities for faculty to acquire proficiency in critical appraisal.59,60

When asked if they would be interested in receiving a monthly EBM journal club curriculum that included relevant articles, expert critical appraisals of the articles, and worksheets with answer keys, most responded affirmatively.61 We believe that providing such content could address some of the barriers to EBM dissemination that we have described.16,62,63

Limitations

The survey was not validated, although the instrument has content and face validity. We surveyed a small proportion of CORD members and cannot be certain that our sampling is representative of the organizational whole. The CORD membership is 737 individuals from 221 programs, including allopathic and osteopathic programs, along with fellowships.64 Our survey consisted of 98 respondents. The small sample size means that a Type II error is possible. Furthermore, survey respondents likely represent a select subset of CORD members likely to attend the annual meeting and participate in educational projects. Additionally, we did not track the number of residency programs that were represented by our individual respondents, so multiple surveys may have represented the same program. Therefore, our results may have a spectrum bias and lack external validity for all of the EM GME community.

Conclusions

Among emergency medicine educators, the attainment of evidence-based medicine proficiency by faculty and resident learners is an expected outcome of graduate medical education training, although the specific domains of expertise differ between faculty and residents. Despite this, only a minority of respondents currently use a formalized curriculum for EBM skills’ development. Journal club directors’ primary evidence-based medicine educational objective for resident learners is to identify secondary peer-reviewed sources, whereas non–journal club directors favor learners’ ability to distinguish significantly flawed research designs. There appears to be a high level of interest in obtaining EBM journal club educational content in a structured format. Measuring the effects of providing journal club curriculum content in conjunction with other evidence-based medicine interventions may warrant further investigation.

The authors thank John A. Ullian, PhD, for his guidance in developing the survey instrument. Additionally, we thank Cora Sias for collecting all survey responses to strip them of all identifiers prior to releasing them to the investigators. We acknowledge Teresa Vallera and Andrew Worster, MD, for their time and assistance in putting the Best Evidence in Emergency Medicine manual together for conference attendees. We also offer our appreciation to Francis Counselman, MD, and Lawrence Lewis, MD, for their insightful review of earlier versions of the manuscript.

Ancillary