An acute care skills evaluation for graduating medical students: a pilot study using clinical simulation


  • Received 27 November 2001; editorial comments to authors 25 February 2002; accepted for publication 10 May 2002

John R. Boulet, PhD, Test Development and Research, Clinical Skills Assessment, Educational Commission for Foreign Medical Graduates, 3624 Market Street, Philadelphia, Pennsylvania 19104–2685, USA. Tel.: 00 1 215 823 2227; Fax: 00 1 215 383 3309; E-mail:


Purpose  This investigation aimed to explore the measurement properties of scores from a patient simulator exercise.

Methods  Analytic and holistic scores were obtained for groups of medical students and residents. Item analysis techniques were used to explore the nature of specific examinee actions. Interrater reliability was calculated. Scores were contrasted for third year medical students, fourth year medical students and emergency department residents.

Results  Interrater reliabilities for analytic and holistic scores were 0·92 and 0·81, respectively. Based on item analysis, proper timing and sequencing of actions discriminated between low- and high-ability examinees. In general, examinees with more advanced training obtained higher scores on the simulation exercise.

Conclusion  Reliable and valid measures of clinical performance can be obtained from a trauma simulation provided that care is taken in the development and scoring of the scenario.