2024 Theses Doctoral

Constructing Task-Oriented Dialogue Systems with Limited Resources

Qian, Kun

Task-oriented dialogue systems have increasingly become integral to our daily lives. However, collecting dialogue data is notably expensive due to the necessity of human interaction. These systems are used in various applications, such as customer service chatbots, virtual assistants, and automated scheduling tools. Given the critical role of these systems, it is essential to develop methods that can leverage limited resources efficiently, especially in data-driven models like neural networks, which have demonstrated superior performance and widespread adoption. This dissertation proposes systematic approaches to address the limited-data problem in both modeling and data aspects, aiming to enhance the effectiveness and efficiency of task-oriented dialogue systems even when data is scarce.

This dissertation is divided into three main parts. The first part introduces three modeling techniques to tackle limited-data challenges. As the base dialogue model evolves from traditional recurrent neural networks to advanced large language models, we explore meta-learning methods, meta-in-context learning, and pre-training sequentially. Besides modeling considerations, the second part of our discussion emphasizes evaluation benchmarks. We start by discussing our work on correcting MultiWOZ, one of the most popular task-oriented dialogue datasets, which enhances training and provides more accurate evaluations. We also investigate biases within this dataset and propose methods to mitigate them. Additionally, we aim to improve the dataset by extending it to a multilingual dataset, facilitating the development of task-oriented dialogue systems for a global audience. The last part examines how to adapt our methods to real-world applications. We address the issue of database-search-result ambiguity in Meta’s virtual assistants by constructing disambiguation dialogue turns in the training data. Furthermore, we aim to enhance Walmart’s shopping companion by synthesizing high-quality knowledge-based question-answer pairs and constructing dialogue data from the bottom up.

Throughout this dissertation, the consistent focus is on developing effective approaches to building task-oriented dialogue systems with limited resources. Our strategies include leveraging limited data more efficiently, utilizing data from other domains, improving data quality, and distilling knowledge from pre-trained models. We hope our approach will contribute to the field of dialogue systems and natural language processing, particularly in building applications involving real-world limited data and minimizing the need for manual data construction efforts. By addressing these challenges, this dissertation aims to lay the groundwork for creating more robust, efficient, and scalable task-oriented dialogue systems that better serve diverse user needs across various industrial applications.