2026 Theses Doctoral

Newton-Raphson Approximate Re-optimization in High Dimensions: Theory and Applications to Machine Unlearning, Risk Estimation and Data Attribution

Zou, Haolin

This dissertation develops a unified framework for approximate re-optimization in high-dimensional statistical models using a few Newton–Raphson iterations. Motivated by settings where the objective function is slightly perturbed, such as machine unlearning, cross-validation, and data attribution, we study how to update model parameters efficiently without retraining, while maintaining rigorous guarantees.

Working under the proportional high-dimensional regime, we establish finite sample error bounds for Newton-based approximations and characterize the number of iterations required for statistical accuracy. We apply this framework to three problems: (i) certified machine unlearning for smooth objectives via Newton updates with calibrated noise, (ii) approximate leave-one-out cross-validation for non-smooth regularizers, and (iii) bias correction of influence functions, leading to a consistent estimator termed Newfluence. Overall, the thesis demonstrates that combining second-order approximations with high-dimensional analysis yields a principled and efficient approach to re-optimization in modern learning systems.