2021 Theses Doctoral

Robust and Interpretable Sequential Decision-Making for Healthcare

Grand-Clement, Julien

Markov Decision Processes (MDP) is a common framework for modeling sequential decision-making problems, with applications ranging from inventory and supply chains to healthcare applications, autonomous driving and solving repeated games. Despite its modeling power, two fundamental challenges arise when using the MDP framework in real-worldapplications. First, the optimal decision rule may be highly dependent on the MDP parameters (e.g., transition rates across states and rewards for each state-action pair). When the parameters are miss-estimated, the resulting decision rule may be suboptimal when deployed in practice.

Additionally, the optimal policies computed by state-of-the-art algorithms may not be interpretable and can be seen as a black-box. Among other reasons, this is problematic as the policy may not be understandable for the people that are supposed to operationalize it. In this thesis, we aim to broaden the applicability of the MDP framework by addressing the challenges of robustness and interpretability. In the first part of the thesis, we focus on robustness.

We introduce a novel model for parameter uncertainty in Chapter 2, that is significantly less pessimistic than prior models of uncertainty while enabling the efficient computation of a robust policy. In Chapter 3, we consider a healthcare application, where we focus on proactively transferring patients of a hospital to the Intensive Care Unit, to ameliorate the overall survival rates and patients’ flow. In the second part of this thesis, we focus on interpretable algorithms, with an emphasis on the application to find novel triage protocols for ventilator allocations for COVID-19 patients.

In Chapter 4, we introduce a simulation model to estimate the performance of the official New York State (NYS) triage protocol at various levels of shortages of ventilators, using a real data set of patients intubated during Spring 2020 because of COVID-19 complications. In Chapter 5, we introduce our algorithmic framework for computing interpretable (tree) policies and apply our methods to learn novel triage protocols.