This research was partially funded by the Graduate Record Examinations Board. The opinions expressed in this article are not necessarily those of the sponsor.
Evaluating an Automatically Scorable, Open-Ended Response Type for Measuring Mathematical Reasoning in Computer-Adaptive Tests
Article first published online: 12 SEP 2005
Journal of Educational Measurement
Volume 34, Issue 2, pages 162–176, June 1997
How to Cite
Bennett, R. E., Steffen, M., Singley, M. K., Morley, M. and Jacquemin, D. (1997), Evaluating an Automatically Scorable, Open-Ended Response Type for Measuring Mathematical Reasoning in Computer-Adaptive Tests. Journal of Educational Measurement, 34: 162–176. doi: 10.1111/j.1745-3984.1997.tb00512.x
We gratefully acknowledge the key contributions of Ken Berger, Dave Bostain, Daryl Ezzo, Jutta Levin, Alex Vasilev, and the Mathematical Reasoning Team in developing the ME response type, writing ME items, and collecting and analyzing the response data.
Requests for reprints should be sent to the first author.
- Issue published online: 12 SEP 2005
- Article first published online: 12 SEP 2005
The first generation of computer-based tests depends largely on multiple-choice items and constructed-response questions that can be scored through literal matches with a key. This study evaluated scoring accuracy and item functioning for an open-ended response type where correct answers, posed as mathematical expressions, can take many different surface forms. Items were administered to 1,864 participants in field trials of a new admissions test for quantitatively oriented graduate programs. Results showed automatic scoring to approximate the accuracy of multiple-choice scanning, with all processing errors stemming from examinees improperly entering responses. In addition, the items functioned similarly in difficulty, item-total relations, and male-female performance differences to other response types being considered for the measure.