2026 Theses Doctoral

Topics on Robust and Efficient Statistical Learning

Huang, Yian

This dissertation includes several works on the robustness and efficiency of statistical learning.

First, we study the robustness of the representation-based multi-task learning (MTL). We consider the practical scenarios where there is unknown and potentially substantial contamination, while also allowing for heterogeneity among inlier tasks. We introduce a Robust and Adaptive Spectral (RAS) method that can learn the shared inlier representation effectively. Theoretically, we provide non-asymptotic error bounds for both the learned representation and the per-task parameters. These bounds adapt to inlier task similarity and outlier task structure, and guarantee that RAS does not suffer from negative transfer. Our method is also extended to the transfer learning setting with corresponding theoretical guarantees for the target task. Extensive experiments confirm our theory, showcasing the robustness and adaptivity of our proposed method, and its superior performance in a large range of regimes.

Second, we propose the quasi-Monte Carlo (QMC) features to accelerate kernel methods. It is shown that for a large class of kernels, the proposed QMC methods achieve an improved approximation error for estimation of the kernel function and the associated integral operator. In the kernel ridge regression, QMC features also exhibit substantial benefits. We show that fewer random features suffice to guarantee the same convergence rate of the excess risk. Extensive empirical results show the superior performance of our proposed QMC features, especially in low-dimensional regimes.

Third, we propose the randomized quasi-Monte Carlo (RQMC) methods as a tool for efficient kernel-based statistical learning, in both low and moderately high-dimensional regimes. Compared to the classical Monte Carlo (MC) approach, the RQMC methods improve the approximation error bound, matching the rate achieved by QMC methods. Besides, average error bounds are established for RQMC features. In the kernel ridge regression, RQMC features exhibit substantial improvements in computational efficiency over MC features while preserving the same statistical accuracy. Extensive empirical results demonstrate that RQMC methods maintain stable performance in both low, and moderately high-dimensional settings where the QMC methods may suffer from performance degradation.