Academic Commons

Theses Doctoral

Studies of Extensions of HRM-SDT for Constructed Responses

Zhou, Xiaoliang

This research examines an ordered perception rater model, an extension of the equal perception signal detection theory (SDT) latent class rater model. The expectation-maximization algorithm and the Newton-Raphson algorithm are used to estimate parameters. Four simulation studies are conducted to answer three research questions.
Simulation studies 1 and 2 fit correct models to the data. Simulation study 1 generates one hundred data sets from the equal perception rater model, both with fully-crossed design and BIB design, and both without and with rater effects, and fits the equal perception model. Parameter recovery is excellent for fully-crossed design and reasonable for BIB design, and all rater effects are detected. Simulation study 2 generates one hundred simulated data sets from the ordered perception model, both with fully-crossed design and BIB design, and both without and with rater effects, and fits the ordered perception rater model. Although parameter recovery is biased for some parameters in the BIB design, all rater effects are recovered.
Simulation studies 3 and 4 fit wrong models to the data. Simulation study 3 fits equal perception models to the fully-crossed and BIB ordered perception data sets generated in simulation study 2. All rater effects are revealed, although rater effects are distorted to some extent in the BIB design. Simulation study 4 fits ordered perception models to the fully-crossed and BIB equal perception data sets generated in study 1. All rater effects are recovered.
Using essay scores from a large-scale language test, an empirical study is conducted. Both the equal and the ordered perception models are fitted. Information criteria favor the equal perception model.

Files

  • thumnail for Zhou_columbia_0054D_15235.pdf Zhou_columbia_0054D_15235.pdf application/pdf 1.34 MB Download File

More About This Work

Academic Units
Measurement and Evaluation
Thesis Advisors
DeCarlo, Lawrence
Degree
Ph.D., Teachers College, Columbia University
Published Here
April 30, 2019
Academic Commons provides global access to research and scholarship produced at Columbia University, Barnard College, Teachers College, Union Theological Seminary and Jewish Theological Seminary. Academic Commons is managed by the Columbia University Libraries.