Abstract
Goal-oriented dialogue systems face a trade-off between fluent language generation and task-specific control. While supervised learning with large language models is capable of producing realistic text, how to steer such responses towards completing a specific task without sacrificing language quality remains an open question. In this work, we formulate goal-oriented dialogue as a partially observed Markov decision process, interpreting the language model as a representation of both the dynamics and the policy. This view allows us to extend techniques from learning-based control, such as task relabeling, to derive a simple and effective method to finetune language models in a goal-aware way, leading to significantly improved task performance. We additionally introduce a number of training strategies that serve to better focus the model on the task at hand. We evaluate our method, Context-Aware Language Models (CALM), on a practical flight-booking task using AirDialogue. Empirically, CALM outperforms the state-of-the-art method by 7% in terms of task success, matching human-level task performance.- Anthology ID:
- 2022.findings-naacl.181
- Volume:
- Findings of the Association for Computational Linguistics: NAACL 2022
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2351–2366
- Language:
- URL:
- https://aclanthology.org/2022.findings-naacl.181
- DOI:
- 10.18653/v1/2022.findings-naacl.181
- Cite (ACL):
- Charlie Snell, Sherry Yang, Justin Fu, Yi Su, and Sergey Levine. 2022. Context-Aware Language Modeling for Goal-Oriented Dialogue Systems. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2351–2366, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- Context-Aware Language Modeling for Goal-Oriented Dialogue Systems (Snell et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2022.findings-naacl.181.pdf