Get access

Setting Standards for English Foreign Language Assessment: Methodology, Validation, and a Degree of Arbitrariness

Authors


Simon P. Tiffin-Richards, Institute for Educational Quality Improvement (IQB), Humboldt University, Unter den Linden 6, 10099 Berlin, Germany; tiffinr@zedat.fu-berlin.de. Hans Anand Pant, Institute for Educational Quality Improvement (IQB), Humboldt University, Berlin, Germany. Olaf Köller, Leibniz-Institute for Science Education (IPN), University of Kiel, Germany.

Abstract

Cut-scores were set by expert judges on assessments of reading and listening comprehension of English as a foreign language (EFL), using the bookmark standard-setting method to differentiate proficiency levels defined by the Common European Framework of Reference (CEFR). Assessments contained stratified item samples drawn from extensive item pools, calibrated using Rasch models on the basis of examinee responses of a German nationwide assessment of secondary school language performance. The results suggest significant effects of item sampling strategies for the bookmark method on cut-score recommendations, as well as significant cut-score judgment revision over cut-score placement rounds. Results are discussed within a framework of establishing validity evidence supporting cut-score recommendations using the widely employed bookmark method.

Get access to the full text of this article

Ancillary