2026 Theses Doctoral

Efficient Inference and Decision-Making for Stochastic Systems

Wang, Zitong

Sampling-based computation, such as Monte Carlo simulation, stochastic optimization, and reinforcement learning, is central to modern operations research and machine learning, yet the statistical noise inherent in sampling creates persistent challenges for both estimation accuracy and computational efficiency. This dissertation develops new methodology that addresses these challenges from two complementary angles. The first two chapters work within the sampling paradigm, developing statistical tools that extract more information from each sample: computationally cheap inference for stochastic optimization and variance reduction for nonlinear functionals. The final chapter develops a fully deterministic approach to an important class of stochastic control problems, directly avoiding the noise and computational burden associated with sampling.

Chapter 2 addresses a fundamental challenge in sampling-based optimization: once a sampling-based algorithm has produced a point estimate, how can one quantify its uncertainty without incurring substantial additional computation? The chapter introduces the cheap bootstrap for fast uncertainty quantification of stochastic gradient descent. Existing approaches to SGD inference suffer from high computational cost (Hessian estimation, large numbers of bootstrap replicates) or require delicate hyperparameter tuning. The cheap bootstrap exploits the joint distribution between original and resampled estimates to construct a pivotal Student 𝑡-statistic, yielding asymptotically exact confidence intervals with as few as one or two bootstrap replicates. Two deployment variants are developed: the Cheap Offline Bootstrap (COfB) for stored datasets and the Cheap Online Bootstrap (COnB) for streaming data.

Chapter 3 extends control variate methodology beyond sample means. The first part develops a general weighted-empirical-distribution approach for applying control variates to nonlinear statistical functionals, including quantiles, conditional value-at-risk, and solutions to stochastic optimization problems. By reformulating the control variate estimator as a reweighted empirical distribution, the method bypasses the computation of influence functions and automatically calibrates the optimal coefficient through the weights, achieving the same asymptotic variance reduction as an oracle procedure. The second part specializes these ideas to the variance of a conditional expectation, a quantity central to input uncertainty quantification in stochastic simulation. By constructing control variates that target the centered-quadratic structure of the nested-simulation estimator, substantial variance reduction is achieved while preserving unbiasedness.

While the first two chapters develop statistical tools to cope with sampling noise, Chapter 4 takes a fundamentally different approach for problems where sufficient structural information is available. Motivated by the design and control of queueing systems with non-stationary arrivals, general service-time distributions, and chance constraints, the chapter introduces QPLEX Decision Processes (QDPs), which integrate the QPLEX modeling paradigm into a nonlinear Markov decision framework. Since QPLEX uses nonlinear transition probabilities on an orders-of-magnitude smaller state space, QDPs circumvent the curse of dimensionality associated with general service times. A policy gradient framework for this nonlinear setting yields a backward iterative scheme that computes exact gradients deterministically, complementing the QPLEX forward scheme for performance evaluation. Optimization is addressed through a natural-gradient-inspired algorithm with block-diagonal Fisher approximations, leading to an exponentiated Q-ascent method that converges to the exact natural gradient near deterministic policies. Experiments on queueing systems with up to 10¹¹ states demonstrate near-optimal performance in seconds and decisive advantages over both exact solvers and simulation-based reinforcement learning.