Handbook of Diagnostic Classification Models

This handbook provides an overview of major developments around diagnostic classification models (DCMs) with regard to modeling, estimation, model checking, scoring, and applications. It brings together not only the current state of the art, but also the theoretical background and models developed for diagnostic classification. The handbook also offers applications and special topics and practical guidelines how to plan and conduct research studies with the help of DCMs. Commonly used models in educational measurement and psychometrics typically assume a single latent trait or at best a small number of latent variables that are aimed at describing individual differences in observed behavior. While this allows simple rankings of test takers along one or a few dimensions, it does not provide a detailed picture of strengths and weaknesses when assessing complex cognitive skills. DCMs, on the other hand, allow the evaluation of test taker performance relative to a potentiallylarge number of skill domains. Most diagnostic models provide a binary mastery/non-mastery classification for each of the assumed test taker attributes representing these skill domains. Attribute profiles can be used for formative decisions as well as for summative purposes, for example in a multiple cut-off procedure that requires mastery on at least a certain subset of skills. The number of DCMs discussed in the literature and applied to a variety of assessment data has been increasing over the past decades, and their appeal to researchers and practitioners alike continues to grow. These models have been used in English language assessment, international large scale assessments, and for feedback for practice exams in preparation of college admission testing, just to name a few. Nowadays, technology-based assessments provide increasingly rich data on a multitude of skills and allow collection of data with respect to multiple types of behaviors. Diagnostic models can be understood as an ideal match for these types of data collections to provide more in-depth information about test taker skills and behavioral tendencies.

Auteur
Matthias von Davier is Distinguished Research Scientist at the National Board of Medical Examiners (NBME), in Philadelphia, Pennsylvania. Until 2016, he was a senior research director in the Research & Development Division at Educational Testing Service (ETS), and co-director of the center for Global Assessment at ETS, leading psychometric research and operations of the center. He earned his Ph.D. at the University of Kiel, Germany, in 1996, specializing in psychometrics. In the Center for Advanced Assessment at NBME, he works on psychometric methodologies for analyzing data from technology-based high-stakes assessments. He is one of the editors of the Springer journal Large Scale Assessments in Education, which is jointly published by the International Association for the Evaluation of Educational Achievement (IEA) and ETS. He is also editor-in-chief of the British Journal of Mathematical and Statistical Psychology (BJMSP), and co-editor of the Springer book series Methodology of Educational Measurement and Assessment. Dr. von Davier received the 2006 ETS Research Scientist award and the 2012 NCME Brad Hanson Award for contributions to educational measurement. His areas of expertise include topics such as item response theory, latent class analysis, diagnostic classification models, and, more broadly, classification and mixture distribution models, computational statistics, person-fit, item-fit, and model checking, hierarchical extension of models for categorical data analysis, and the analytical methodologies used in large scale educational surveys.

Dr. Lee is an Associate Professor in the program of Measurement, Statistics & Evaluation, in the Department of Human Development at Teachers College, Columbia University. She received her Ph.D. in Quantitative Methods at the University of Wisconsin-Madison, with a minor in Statistics. Her research interests are primarily on psychometric approaches to solving practical problems in educational and psychological testing. Her areas of expertise include topics such as development and applications of diagnostic classification models, item response theory, latent class models, and analytical methodologies used in large scale assessments. In addition to her own research, Dr. Lee collaborates on various projects on the use of latent variable models for purposes of scale development/test construction and for validity studies.

Contenu

1. Introduction: From Latent Class Analysis to DINA and Beyond; Matthias von Davier and Young-Sun Lee.- PART 1: Approaches to Cognitive Diagnosis.- 2. Nonparametric Item Response Theory and Mokken Scale Analysis, with Relations to Latent Class Models and Cognitive Diagnostic Models; Andries van der Ark, Gina Rossi, and Klaas Sijtsma.- 3. The Reparameterized Unified Model System: A Diagnostic Assessment Modeling Approach; William F. Stout, Robert Henson, Lou DiBello, and Benjamin Shear.- 4. Bayesian Networks; Russell Almond and Diego Zapata.- 5. Nonparametric Classification Models; Chia-Yi Chiu and Hans-Friedrich Kohn.- 6. General Diagnostic Model (GDM); Matthias von Davier.- 7. Generalized Deterministic Inputs, Noisy and Gate Model (G-DINA); Jimmy de la Torre and Nathan Minchen.- 8. Loglinear Cognitive Diagnostic Model (LCDM); Robert Henson and Jonathan Templin.- 9. Diagnostic Modeling of Skill Hierarchies and Cognitive Process with MLTM-D; Susan Embretson.- 10. Explanatory Diagnostic Models; Yoon Soo Park and Young-Sun Lee.- 11. Insights from Reparametrized DINA and Beyond; Lawrence T. DeCarlo.- PART 2: Special Topics.- 12. Q Matrix Learning via Latent Variable Selection and Identifiability; Jingchen Liu and Hyeon-Ah Kang.- 13. Global Model and Item-level Fit Indices; Zhaungzhuang Han and Matthew Johnson.- 14. Exploratory Data Analysis and Cognitive Diagnostic Model; Yunxiao Chen and Xiaoou Li.- 15. CDM-CAT; Xiaofeng Yu, Ying Cheng, and Hua Hua Chang.- 16. Identifiability and Cognitive Diagnostic Model; Gongjun Xu.- 17. Classification Consistency and Reliability; Sandip Sinharay and Matthew Johnson.- 18. Differential Item Functioning in CDM; Xuelan Qiu, Xiaomin Li, and Wen-Chung Wang.- 19. Parameter Invariance and Skill Attribute Continuity in DCMs: Bifactor MIRT as an Appealing and Related Alternative; Daniel Bolt.- PART 3: Applications.- 20. Application of CDMs to Process Data Analysis; Hong Jiao, Dandan Liao, and Peida Zhan.- 21. Application of CDMs to Learning Systems; Benjamin Deonovic, Pravin Chopade, Michael Yudelson, Jimmy de la Torre, Alina von Davier.- 22. CDMs in Vocational Education; Stephan Abele and Matthias von Davier.- 23. Analyzing Large Scale Assessment Data with Diagnostic Models ; Xueli Xu and Matthias von Davier.- 24. Reduced Reparameterized Unified Model Applied to Learning Spatial Reasoning Skills; Susu Zhang, Jeff Douglas, Shiyu Wang, and Steve Culpepper.- 25. How to Conduct a Study with Diagnostic Models ; Young-Sun Lee and Diego Luna Bazaldua.- PART 4: Software, Data, and Tools.- 26. The R package CDM for Diagnostic Modeling; Alexander Robitzsch and Ann Cathrice George.- 27. Diagnostic Classification Modeling with flexMIRT; Li Cai and Carrie Houts.- 28. Using Mplus to Estimate the Log-Linear Cognitive Diagnosis Model; Meghan Sullivan, Jesse Pace, and Jonathan Templin.- 29. The GDINA R-package; Wenchao Ma.- 30. GDM software mdltm including Parallel EM algorithm; Lale Khorramdel, Hyo Jeong Shin, and Matthias von Davier.- 31. Estimating CDMs using MCMC; Xiang Liu and Matthew Johnson.