2025 Theses Doctoral

Propensity Score Estimation with Multilevel Data: A Monte Carlo Study

Zhang, Tianyang

Propensity score (PS) methods have become a cornerstone for estimating causal effects in observational studies. While significant advancements have been made in understanding PS estimation, most studies focus on either parametric models tailored for multilevel data or machine learning methods applied to single-level datasets. There remains a critical gap in systematically comparing these approaches for estimating PS in multilevel observational data.

This dissertation addresses this gap by evaluating the performance of parametric models (e.g., generalized linear mixed models) and machine learning techniques (e.g., Bayesian Additive Regression Trees [BART] and Generalized Boosted Models [GBM]) through a Monte Carlo simulation framework.

The study simulated multilevel datasets under varying conditions, including sample sizes, intraclass correlation (ICC), and the complexity of data relationships (linear/additive versus non-linear/non-additive). Propensity scores were estimated using six methods, ranging from traditional parametric approaches to advanced machine learning techniques. Treatment effects were subsequently estimated via inverse probability weighting, with performance metrics including bias, root mean square error, and covariate balance.

Results demonstrated that parametric methods were robust when the true PS model adhered to linear and additive assumptions, aligning with prior research. However, in non-linear or non-additive scenarios, BART and GBM outperformed traditional models, offering superior covariate balance and reduced bias, albeit with overfitting risks in small samples. These findings provide practical guidance for researchers, suggesting that the choice of PS estimation methods should align with data characteristics. Future research should explore alternative PS applications and real-world scenarios with variable cluster sizes and treatment proportions.