2025 Theses Doctoral

Advancing Dialogue Systems for Social Good: Human-Robot Interaction, Knowledge Integration, and Real-World Applications

Li, Yu

Dialogue systems are increasingly being developed to address societal needs, offering opportunities for meaningful interactions that contribute to social good. This thesis explores the development of dialogue systems by addressing key challenges, including mitigating the risks of self-anthropomorphism to ensure ethical and trustworthy interactions, integrating knowledge to enhance accuracy and reliability, and deploying these systems in practical applications such as anti-scam systems and curriculum-aligned educational chatbots.

We begin by examining self-anthropomorphism in dialogue systems, where AI mimics human identities and behaviors, potentially leading to misplaced trust and unrealistic expectations. Our initial work, Robots Don’t Cry, provides an analytical foundation by assessing how feasible human-like utterances appear to users across diverse dialogue datasets. These findings reveal that many anthropomorphic responses are perceived as inappropriate or misleading, regardless of the AI's embodiment. Building on these insights, our follow-up work, Pix2Persona, introduces a dataset that systematically models and transforms dialogue responses along the anthropomorphism spectrum. This enables dialogue systems to dynamically adjust their persona and language style, aligning with ethical considerations and user expectations across different applications.

Next, we tackle the challenge of ensuring that dialogue systems provide reliable, knowledge-grounded responses, an essential component of socially beneficial interactions. Many systems struggle to generalize across unseen topics or to integrate diverse external information sources effectively. To address this, we develop a unified knowledge-based framework that consolidates resources such as wikis, knowledge graphs, and dictionaries into a cohesive input format for language models. This approach enhances the contextual accuracy and robustness of responses across multiple domains, enabling dialogue systems to support real-world applications more safely and effectively.

We demonstrate the practical impact of these approaches through two dialogue system applications: an anti-scam system designed to help users resist fraudulent schemes, and a curriculum-driven EduBot that provides structured conversational practice aligned with textbook content. In addition, we explore the use of vision-language models to support dialogue systems for people with blindness and low vision. We evaluate these models on fundamental visual reasoning tasks related to navigation, offering insights into their potential and limitations for future multimodal assistive systems.

Through these contributions, this thesis illustrates how dialogue systems can be responsibly designed, knowledge-grounded, and practically deployed to advance social good and improve lives.