A Comparative Study of IRT Fixed Parameter Calibration Methods

Abstract

This article provides technical descriptions of five fixed parameter calibration (FPC) methods, which were based on marginal maximum likelihood estimation via the EM algorithm, and evaluates them through simulation. The five FPC methods described are distinguished from each other by how many times they update the prior ability distribution and by how many EM cycles they use. Specifically, the five FPC methods included no prior weights updating and one EM cycle (NWU‐OEM) or multiple EM cycles (NWU‐MEM), one prior weights updating and one EM cycle (OWU‐OEM) or multiple EM cycles (OWU‐MEM), and multiple weights updating and multiple EM cycles (MWU‐MEM) methods. All the five FPC methods were evaluated in terms of recovery of the underlying ability distribution and item parameters. An important factor in the simulation was three different ability (normal) distributions—N(0, 1), N(0.5, 1.2²), and N(1, 1.4²)—for FPC groups, with the fixed item parameters obtained with a reference N(0, 1) group. Only the MWU‐MEM method appeared to perform properly under all the three distributions. Under the N(0, 1) distribution, the NWU‐MEM and OWU‐MEM methods also appeared to perform properly. Under the N(0.5, 1.2²), and N(1, 1.4²) distributions, however, the four methods other than the MWU‐MEM method resulted in some or severe under‐estimation in the recovery.