Using rasch measurement to score, evaluate, and improve examinations in an anatomy course

Authors

  • Kenneth D. Royal,

    Corresponding author
    1. Office of Medical Education, University of North Carolina, Chapel Hill School of Medicine, Chapel Hill, North Carolina
    2. Department of Family Medicine, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, North Carolina
    • Correspondence to: Dr. Kenneth D. Royal; Office of Medical Education, University of North Carolina at Chapel Hill, 4200 Medical Biomolecular Research Building, 111 Mason Farm Road Chapel Hill, NC 27599-7545, USA. E-mail: Kenneth_Royal@med.unc.edu

    Search for more papers by this author
  • Kurt O. Gilliland,

    1. Department of Cell Biology and Physiology, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, North Carolina
    Search for more papers by this author
  • Edward T. Kernick

    1. Department of Cell Biology and Physiology, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, North Carolina
    Search for more papers by this author

Abstract

Any examination that involves moderate to high stakes implications for examinees should be psychometrically sound and legally defensible. Currently, there are two broad and competing families of test theories that are used to score examination data. The majority of instructors outside the high-stakes testing arena rely on classical test theory (CTT) methods. However, advances in item response theory software have made the application of these techniques much more accessible to classroom instructors. The purpose of this research is to analyze a common medical school anatomy examination using both the traditional CTT scoring method and a Rasch measurement scoring method to determine which technique provides more robust findings, and which set of psychometric indicators will be more meaningful and useful for anatomists looking to improve the psychometric quality and functioning of their examinations. Results produced by the more robust and meaningful methodology will undergo a rigorous psychometric validation process to evaluate construct validity. Implications of these techniques and additional possibilities for advanced applications are also discussed. Anat Sci Educ 7: 450–460. © 2014 American Association of Anatomists.

Ancillary