2026 Theses Doctoral

Intersections of Machine Learning and Decision Making

Cai, Tianhui

This dissertation studies intersections of machine learning and decision-making. These intersections include important questions that arise when model outputs inform decisions, or model failures require a response. It considers three such problems: deciding how to respond when distribution shift degrades model performance, estimating the effects of interventions using machine learning methods, and modeling uncertainty to guide active exploration in sequential decision problems.

Prediction models often perform worse when deployed on target distributions different from the training distribution. Because different kinds of distribution shift require different solutions, understanding why model performance dropped is necessary for understanding how best to improve the model. We develop Distribution Shift Decomposition (DISDE), a diagnostic that attributes changes in performance to different kinds of distribution shift. We demonstrate how DISDE can be used to improve model performance while minimizing the cost of additional data collection.

Causal inference is central to scientific decision-making. However, choosing an estimator can be challenging: When there is limited overlap between treatment and control, estimators with desirable asymptotic properties (e.g., AIPW for estimating the average treatment effect) require ad-hoc adjustments in order to produce stable estimates. In contrast, simple plug-in estimators are stable but lack important asymptotic properties. We propose a new estimation framework based on constrained optimization that achieves the best of both worlds: it produces stable plug-in estimates with desirable asymptotic properties, without additional heuristics. Our approach is versatile and can leverage a variety of modern ML methods. Empirically, it outperforms standard debiased methods when there is limited overlap, and exhibits robust performance across a variety of settings.

In sequential decision-making settings with uncertainty, it is important to actively gather information to resolve uncertainty about which actions are best. While classical Bayesian approaches to modeling uncertainty for principled exploration can be difficult to scale to modern machine learning settings, we propose reframing uncertainty as arising from unobserved outcomes rather than unobservable latent parameters, enabling generative sequence models to drive principled exploration. We prove that offline prediction quality controls online decision-making performance, providing a scalable bridge between sequence modeling and classical exploration algorithms, and experiments show our approach scales to a challenging news recommendation setting.