Get access

A comparison of statistical selection strategies for univariate and bivariate log-linear models

Authors

  • Tim Moses,

    Corresponding author
    1. Educational Testing Service, Princeton, New Jersey, USA
      Correspondence should be addressed to Dr Tim Moses, Educational Testing Service, Rosedale Road, MS 03-P, Princeton, NJ 08541, USA (e-mail: tmoses@ets.org).
    Search for more papers by this author
  • Paul W. Holland

    1. Educational Testing Service, Princeton, New Jersey, USA
    Search for more papers by this author

Correspondence should be addressed to Dr Tim Moses, Educational Testing Service, Rosedale Road, MS 03-P, Princeton, NJ 08541, USA (e-mail: tmoses@ets.org).

Abstract

In this study, eight statistical selection strategies were evaluated for selecting the parameterizations of log-linear models used to model the distributions of psychometric tests. The selection strategies included significance tests based on four chi-squared statistics (likelihood ratio, Pearson, Freeman–Tukey, and Cressie–Read) and four additional strategies (Akaike information criterion (AIC), Bayesian information criterion (BIC), consistent Akaike information criterion (CAIC), and a measure attributed to Goodman). The strategies were evaluated in simulations for different log-linear models of univariate and bivariate test-score distributions and two sample sizes. Results showed that all eight selection strategies were most accurate for the largest sample size considered. For univariate distributions, the AIC selection strategy was especially accurate for selecting the correct parameterization of a complex log-linear model and the likelihood ratio chi-squared selection strategy was the most accurate strategy for selecting the correct parameterization of a relatively simple log-linear model. For bivariate distributions, the likelihood ratio chi-squared, Freeman–Tukey chi-squared, BIC, and CAIC selection strategies had similarly high selection accuracies.

Get access to the full text of this article

Ancillary