2024 Theses Doctoral

On New and Improved Measures for Item Analysis with Signal Detection Theory

Lee, Rachel

Classical item analysis (CIA) entails summarizing items based on two key attributes: item difficulty and item discrimination, defined as the proportion of examinees answering correctly and the difference in correctness between high and low scorers. Recent insights reveal a direct link between these measures and aspects of signal detection theory (SDT) in item analysis, offering modifications to traditional metrics and introducing new ones to identify problematic items (DeCarlo, 2023).

The SDT approach involves extending Luce's choice model (1959) using a mixture framework, with mixing occurring within examinees rather than across them, reflecting varying latent knowledge states (know or don't know) across items. This implies a 'true' split (know/don't know) enabling straightforward discrimination and difficulty measures, lending theoretical support to the conventional item splitting approach. DeCarlo (2023) demonstrated improved measures and item screening using simple median splits, motivating this study to explore enhanced measures via refined splits. This study builds on these findings, refining CIA and SDT measures by integrating additional information like response time and item scores using latent class and cluster models.