Teaching and evaluating first and second year medical students' practice of evidence-based medicine


Robert Holloway MD, MPH, Department of Neurology, University of Rochester, 1351 Mount Hope Avenue, Suite 220, Rochester, New York 14620, USA. Tel: 00 1 585 275 1018; Fax: 00 1 585 461 3554; E-mail: Robert.Holloway@ctcc.rochester.edu


Purpose  To implement an evidence-based medicine (EBM) curriculum for Year 1 and 2 medical students, and to develop a method to evaluate their practice of EBM in discrete and relevant worksteps.

Methods  For the 100 students entering Year 1 of their medical education in 2000, we implemented a curriculum with 25–30 student contact hours of EBM instruction which used a variety of teaching formats and spanned the first and second years of their training. We developed an evaluation module that assessed the following 5 steps in the practice of EBM: generating well built questions; searching for evidence; critical appraisal; applying the evidence, and self-evaluation. We tested 2 different versions of the test module 3-months apart with the same cohort of second year students, and correlated their scores on the second module with examination components of a comprehensive assessment. We obtained feedback from the students regarding the EBM curriculum and evaluation method.

Results  Each test module took 2–4 hours to complete and 5–8 minutes to grade. There was moderate test−retest reliability for the total test scores (r = 0.35, P < 0.001). Step 1 scores correlated with the mock board examination scores (r = 0.23, P = 0.05). Step 2 scores correlated with the peer assessment factor ‘work habits’ (r = 0.24, P = 0.02), and Step 3 scores correlated with clinical reasoning exercises (r = 0.31, P = 0.002). Step 4 scores lacked test−retest reliability and did not correlate with components of the comprehensive assessment. The majority of students felt there was too much focus on EBM during the first 2 years of the curriculum and they rated the EBM test module the lowest rated component of the comprehensive assessment.

Conclusions  Although we have demonstrated preliminary reliability and validity of a new evaluation instrument that assess the domains of scientific knowledge, work habits and reasoning skills required in the practice of EBM, many of the correlations were weak, and we remain in the very early stages of determining if, when and how EBM instruction should occur in medical education.