2026 Theses Doctoral

Modeling Empathy and Human Conversation Dynamics

Chen, Run

Empathetic communication is essential for supporting human well-being, improving social connectedness, and enabling effective dialogue in emotionally charged situations. As conversational AI systems become embedded in daily life, the ability to recognize, interpret, and respond to human emotion responsibly has become a central goal for the field. Yet, existing approaches often rely on text-only inputs and simplistic approximations of supportiveness, making them insufficient for modeling the nuance and complexity of real human empathy. This dissertation advances a holistic framework for empathetic conversational AI that integrates structural conversational dynamics, multimodal signals from speech, scalable learning with large language models, and proactive safeguards against coercive interaction.

The first component of this work strengthens the foundations of conversational understanding. We address long-standing transcription and timing discrepancies in the widely used Switchboard Dialog Act corpus by producing the Re-Aligned Switchboard Dialog Act (𝗥𝗔𝗦𝘄𝗗𝗔) dataset, improving alignment between lexical and acoustic-prosodic cues and enabling more accurate dialog act classification. We also develop new quantitative analyses of prosodic and lexical entrainment in task-oriented settings, revealing how behavioral coordination serves as a precursor to rapport and mutual understanding.

Building on these foundations, the second component investigates empathy as a more specialized and emotionally collaborative form of conversation. We present 𝗘𝗺𝗽𝗦𝗽𝗲𝗲𝗰𝗵, a speech-based empathy corpus annotated at the segment level to capture acoustic-prosodic markers of supportive behavior—representing one of the first resources in computational empathy centered on spoken communication in English. We further examine empathy’s inherent complexity through multimodal learning and model disagreement analysis, demonstrating how conflicting cues across modalities expose the ambiguity and interpretive demands present in real interactions.

To scale empathetic response generation in the age of large language models, we introduce 𝗦𝗬𝗡𝗧𝗛𝗘𝗠𝗣𝗔𝗧𝗛𝗬, a psychotherapy-informed corpus of 106k empathetic dialogues produced without crowdsourcing. Using Chain-of-Empathy prompting, we show improvements in emotional alignment and contextual appropriateness while enabling customizable and domain-adaptable empathetic behavior. Together, these contributions demonstrate that effective empathy modeling requires recognition, interpretation, and expressive alignment across modalities and conversational context.

The final component addresses a critical ethical challenge: ensuring that emotionally aware AI remains trustworthy and protective of users. We develop speech-based detection of mental manipulation using multi-speaker synthetic audio and few-shot learning, illustrating how systems can identify harmful persuasive strategies and prevent coercive influence in vulnerable moments.

Overall, this dissertation presents a unified path toward empathetic and responsible conversational AI: from foundational interaction modeling, to scalable development of emotional intelligence, to proactive safety measures that preserve user autonomy. By advancing empathetic and conversational AI research resources, multimodal modeling, robust analysis techniques, and ethical frameworks, this work contributes essential scientific and practical foundations for conversational technologies that are not only more capable and adaptive, but genuinely supportive of human emotional well-being.