Academic Commons

Conference Objects

A First Step Towards Behavioral Coaching for Managing Stress: A Case Study on Optimal Policy Estimation with Multi-stage Threshold Q-learning

Hu, Xinyu; Hsueh, Pei-Yun S.; Chen, Ching-Hua; Diaz, Keith; Cheung, Ying Kuen; Qian, Min

Psychological stress is a major contributor to the adoption of unhealthy behaviors, which in turn accounts for 41% of global cardiovascular disease burden. While the proliferation of mobile health apps has offered promise to stress management, these apps do not provide micro-level feedback with regard to how to adjust one's behaviors to achieve a desired health outcome. In this paper, we formulate the task of multi-stage stress management as a sequential decision-making problem and explore the application of reinforcement learning to provide micro-level feedback for stress reduction. Specifically, we incorporate a multi-stage threshold selection into Q-learning to derive an interpretable form of a recommendation policy for behavioral coaching. We apply this method on an observational dataset that contains Fitbit ActiGraph measurements and self-reported stress levels. The estimated policy is then used to understand how exercise patterns may affect users' psychological stress levels and to perform coaching more effectively.


  • thumnail for Hu AMIA Annu Symp Proc 2017_PMC.pdf Hu AMIA Annu Symp Proc 2017_PMC.pdf application/pdf 1.17 MB Download File

Also Published In

AMIA Annual Symposium Proceedings

More About This Work

Academic Units
Center for Behavioral Cardiovascular Health
Published Here
June 13, 2018
Academic Commons provides global access to research and scholarship produced at Columbia University, Barnard College, Teachers College, Union Theological Seminary and Jewish Theological Seminary. Academic Commons is managed by the Columbia University Libraries.