Obtaining Diagnostic Classification Model Estimates Using Mplus
Abstract
Diagnostic classification models (aka cognitive or skills diagnosis models) have shown great promise for evaluating mastery on a multidimensional profile of skills as assessed through examinee responses, but continued development and application of these models has been hindered by a lack of readily available software. In this article we demonstrate how diagnostic classification models may be estimated as confirmatory latent class models using Mplus, thus bridging the gap between the technical presentation of these models and their practical use for assessment in research and applied settings. Using a sample English test of three grammatical skills, we describe how diagnostic classification models can be phrased as latent class models within Mplus and how to obtain the syntax and output needed for estimation and interpretation of the model parameters. We also have written a freely available SAS program that can be used to automatically generate the Mplus syntax. We hope this work will ultimately result in greater access to diagnostic classification models throughout the testing community, from researchers to practitioners.
Citing Literature
Number of times cited according to CrossRef: 57
- Randall Davies, Assessing Learning Outcomes, Handbook of Research in Educational Communications and Technology, 10.1007/978-3-030-36119-8, (521-546), (2020).
- Shiyu Wang, Yiling Hu, Qi Wang, Bian Wu, Yawei Shen, Martha Carr, The Development of a Multidimensional Diagnostic Assessment With Learning Tools to Improve 3-D Mental Rotation Skills, Frontiers in Psychology, 10.3389/fpsyg.2020.00305, 11, (2020).
- Chun Wang, Jing Lu, Learning Attribute Hierarchies From Data: Two Exploratory Approaches, Journal of Educational and Behavioral Statistics, 10.3102/1076998620931094, (107699862093109), (2020).
- Chen-Wei Liu, Björn Andersson, Anders Skrondal, A Constrained Metropolis–Hastings Robbins–Monro Algorithm for Q Matrix Estimation in DINA Models, Psychometrika, 10.1007/s11336-020-09707-4, (2020).
- Young-Sun Lee, Diego A. Luna-Bazaldua, How to Conduct a Study with Diagnostic Models, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_25, (525-545), (2019).
- Mariana Curi, Geoffrey A. Converse, Jeff Hajewski, Suely Oliveira, undefined, 2019 International Joint Conference on Neural Networks (IJCNN), 10.1109/IJCNN.2019.8852333, (1-8), (2019).
- Mengta Chung, A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model, Journal of Mathematical Psychology, 10.1016/j.jmp.2019.07.002, 93, (102275), (2019).
- Lawrence T. DeCarlo, Insights from Reparameterized DINA and Beyond, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_11, (223-243), (2019).
- Wenchao Ma, Cognitive Diagnosis Modeling Using the GDINA R Package, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_29, (593-601), (2019).
- Alexander Robitzsch, Ann Cathrice George, The R Package CDM for Diagnostic Modeling, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_26, (549-572), (2019).
- William Stout, Robert Henson, Lou DiBello, Benjamin Shear, The Reparameterized Unified Model System: A Diagnostic Assessment Modeling Approach, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_3, (47-79), (2019).
- Sandip Sinharay, Matthew S. Johnson, Measures of Agreement: Reliability, Classification Accuracy, and Classification Consistency, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_17, (359-377), (2019).
- Xiang Liu, Matthew S. Johnson, Estimating CDMs Using MCMC, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_31, (629-646), (2019).
- Robert Henson, Jonathan L. Templin, Loglinear Cognitive Diagnostic Model (LCDM), Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_8, (171-185), (2019).
- Yunxiao Chen, Xiaoou Li, Exploratory Data Analysis for Cognitive Diagnosis: Stochastic Co-blockmodel and Spectral Co-clustering, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_14, (287-306), (2019).
- Zhuangzhuang Han, Matthew S. Johnson, Global- and Item-Level Model Fit Indices, Handbook of Diagnostic Classification Models, 10.1007/978-3-030-05584-4_13, (265-285), (2019).
- Lian Gafar Otaya, Badrun Kartowagiran, Heri Retnawati, THE ESTIMATION OF TEACHING PROFESSION EDUCATION (TPE) PARTICIPANTS’ ABILITY IN INDONESIA: AN ASSESSMENT MODEL WITH PARTIAL SCORING, Problems of Education in the 21st Century, 10.33225/pec/19.77.790, 77, 6, (790-805), (2019).
- Zhehan Jiang, Kevin Walker, Dexin Shi, Applying AdaBoost to Improve Diagnostic Accuracy, Methodology, 10.1027/1614-2241/a000166, 15, 2, (77-87), (2019).
- Xiaofeng Yu, Ying Cheng, Data‐driven Q‐matrix validation using a residual‐based statistic in cognitive diagnostic assessment, British Journal of Mathematical and Statistical Psychology, 10.1111/bmsp.12191, 0, 0, (2019).
- Peida Zhan, Hong Jiao, Kaiwen Man, Lijun Wang, Using JAGS for Bayesian Cognitive Diagnosis Modeling: A Tutorial, Journal of Educational and Behavioral Statistics, 10.3102/1076998619826040, (107699861982604), (2019).
- Peida Zhan, Wenchao Ma, Hong Jiao, Shuliang Ding, A Sequential Higher Order Latent Structural Model for Hierarchical Attributes in Cognitive Diagnostic Assessments, Applied Psychological Measurement, 10.1177/0146621619832935, (014662161983293), (2019).
- Zhehan Jiang, Using the iterative latent-class analysis approach to improve attribute accuracy in diagnostic classification models, Behavior Research Methods, 10.3758/s13428-018-01191-0, (2019).
- Sedat Sen, Ragip Terzi, A Comparison of Software Packages Available for DINA Model Estimation, Applied Psychological Measurement, 10.1177/0146621619843822, (014662161984382), (2019).
- Xue Xing, Using the log-linear cognitive diagnosis model to classify individuals in career and educational assessment, International Journal for Educational and Vocational Guidance, 10.1007/s10775-019-09390-7, (2019).
- Bo Hu, Jonathan Templin, Using Diagnostic Classification Models to Validate Attribute Hierarchies and Evaluate Model Fit in Bayesian Networks, Multivariate Behavioral Research, 10.1080/00273171.2019.1632165, (1-12), (2019).
- Ren Liu, Zhehan Jiang, A general diagnostic classification model for rating scales, Behavior Research Methods, 10.3758/s13428-019-01239-9, (2019).
- Zhehan Jiang, Richard Carter, Using Hamiltonian Monte Carlo to estimate the log-linear cognitive diagnosis model via Stan, Behavior Research Methods, 10.3758/s13428-018-1069-9, 51, 2, (651-662), (2018).
- Peida Zhan, Hong Jiao, Manqian Liao, Yufang Bian, Bayesian DINA Modeling Incorporating Within-Item Characteristic Dependency, Applied Psychological Measurement, 10.1177/0146621618781594, 43, 2, (143-158), (2018).
- Steven Andrew Culpepper, Yinghan Chen, Development and Application of an Exploratory Reduced Reparameterized Unified Model, Journal of Educational and Behavioral Statistics, 10.3102/1076998618791306, 44, 1, (3-24), (2018).
- Meghan Sullivan, Hongling Lao, Jonathan Templin, Diagnostic Measurement, Innovative Research Methodologies in Management, 10.1007/978-3-319-64394-6, (87-109), (2018).
- Matthew S. Johnson, Sandip Sinharay, Measures of Agreement to Assess Attribute‐Level Classification Accuracy and Consistency for Cognitive Diagnostic Assessments, Journal of Educational Measurement, 10.1111/jedm.12196, 55, 4, (635-664), (2018).
- Ren Liu, Zhehan Jiang, Diagnostic Classification Models for Ordinal Item Responses, Frontiers in Psychology, 10.3389/fpsyg.2018.02512, 9, (2018).
- Fu Chen, Yanlou Liu, Tao Xin, Ying Cui, Applying the M2 Statistic to Evaluate the Fit of Diagnostic Classification Models in the Presence of Attribute Hierarchies, Frontiers in Psychology, 10.3389/fpsyg.2018.01875, 9, (2018).
- Marcelo A. da Silva, Ren Liu, Anne C. Huggins-Manley, Jorge L. Bazán, Incorporating the Q-Matrix Into Multidimensional Item Response Theory Models, Educational and Psychological Measurement, 10.1177/0013164418814898, (001316441881489), (2018).
- Zhehan Jiang, Wenchao Ma, Integrating Differential Evolution Optimization to Cognitive Diagnostic Model Estimation, Frontiers in Psychology, 10.3389/fpsyg.2018.02142, 9, (2018).
- Peida Zhan, Wen-Chung Wang, Hong Jiao, Yufang Bian, Probabilistic-Input, Noisy Conjunctive Models for Cognitive Diagnosis, Frontiers in Psychology, 10.3389/fpsyg.2018.00997, 9, (2018).
- Matthew J. Madison, Laine P. Bradshaw, Assessing Growth in a Diagnostic Classification Model Framework, Psychometrika, 10.1007/s11336-018-9638-5, (2018).
- Shiyu Wang, Yan Yang, Steven Andrew Culpepper, Jeffrey A. Douglas, Tracking Skill Acquisition With Cognitive Diagnosis Models: A Higher-Order, Hidden Markov Model With Covariates, Journal of Educational and Behavioral Statistics, 10.3102/1076998617719727, 43, 1, (57-87), (2017).
- Ren Liu, Misspecification of Attribute Structure in Diagnostic Measurement, Educational and Psychological Measurement, 10.1177/0013164417702458, 78, 4, (605-634), (2017).
- Steven Andrew Culpepper, Aaron Hudson, An Improved Strategy for Bayesian Estimation of the Reduced Reparameterized Unified Model, Applied Psychological Measurement, 10.1177/0146621617707511, 42, 2, (99-115), (2017).
- Yinghan Chen, Steven Andrew Culpepper, Yuguo Chen, Jeffrey Douglas, Bayesian Estimation of the DINA Q matrix, Psychometrika, 10.1007/s11336-017-9579-4, 83, 1, (89-108), (2017).
- Kazuhiro YAMAGUCHI, Kensuke OKADA, Recent Developments in Cognitive Diagnostic Models近年の認知診断モデルの展開, Kodo Keiryogaku (The Japanese Journal of Behaviormetrics), 10.2333/jbhmk.44.181, 44, 2, (181-198), (2017).
- Ren Liu, Hong Qian, Xiao Luo, Ada Woo, Relative Diagnostic Profile: A Subscore Reporting Framework, Educational and Psychological Measurement, 10.1177/0013164417740170, (001316441774017), (2017).
- Hans-Friedrich Köhn, Chia-Yi Chiu, A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests, Psychometrika, 10.1007/s11336-016-9536-7, 82, 1, (112-132), (2016).
- Pui-Wa Lei, Hongli Li, Performance of Fit Indices in Choosing Correct Cognitive Diagnostic Models and Q-Matrices, Applied Psychological Measurement, 10.1177/0146621616647954, 40, 6, (405-417), (2016).
- Chia-Yi Chiu, Hans-Friedrich Köhn, Huey-Min Wu, Fitting the Reduced RUM with Mplus: A Tutorial, International Journal of Testing, 10.1080/15305058.2016.1148038, 16, 4, (331-351), (2016).
- Hamdollah Ravand, Application of a Cognitive Diagnostic Model to a High-Stakes Reading Comprehension Test, Journal of Psychoeducational Assessment, 10.1177/0734282915623053, 34, 8, (782-799), (2016).
- Chia-Yi Chiu, Hans-Friedrich Köhn, Yi Zheng, Robert Henson, Joint Maximum Likelihood Estimation for Diagnostic Classification Models, Psychometrika, 10.1007/s11336-016-9534-9, 81, 4, (1069-1092), (2016).
- Laine Bradshaw, Diagnostic Classification Models, The Wiley Handbook of Cognition and Assessment, 10.1002/9781118956588, (297-327), (2016).
- Hongli Li, C. Vincent Hunter, Pui-Wa Lei, The selection of cognitive diagnostic models for a reading comprehension test, Language Testing, 10.1177/0265532215590848, 33, 3, (391-409), (2015).
- Chia-Yi Chiu, Hans-Friedrich Köhn, The Reduced RUM as a Logit Model: Parameterization and Constraints, Psychometrika, 10.1007/s11336-015-9460-2, 81, 2, (350-370), (2015).
- A. C. George, A. Robitzsch, Cognitive Diagnosis Models in R: A didactic, The Quantitative Methods for Psychology, 10.20982/tqmp.11.3.p189, 11, 3, (189-205), (2015).
- Jonathan Templin, Laine Bradshaw, The Use and Misuse of Psychometric Models, Psychometrika, 10.1007/s11336-013-9364-y, 79, 2, (347-354), (2014).
- Jonathan Templin, Laine Bradshaw, Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies, Psychometrika, 10.1007/s11336-013-9362-0, 79, 2, (317-339), (2014).
- Matthias von Davier, The Log‐Linear Cognitive Diagnostic Model (LCDM) as a Special Case of the General Diagnostic Model (GDM), ETS Research Report Series, 10.1002/ets2.12043, 2014, 2, (1-13), (2014).
- Daniel P. Jurich, Laine P. Bradshaw, An Illustration of Diagnostic Classification Modeling in Student Learning Outcomes Assessment, International Journal of Testing, 10.1080/15305058.2013.835728, 14, 1, (49-72), (2013).
- Laine Bradshaw, Jonathan Templin, Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions, Psychometrika, 10.1007/s11336-013-9350-4, 79, 3, (403-425), (2013).




