Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues
Thibault Cordier, Tanguy Urvoy, Fabrice Lefèvre, Lina M. Rojas Barahona
Abstract
Reinforcement learning has been widely adopted to model dialogue managers in task-oriented dialogues. However, the user simulator provided by state-of-the-art dialogue frameworks are only rough approximations of human behaviour. The ability to learn from a small number of human interactions is hence crucial, especially on multi-domain and multi-task environments where the action space is large. We therefore propose to use structured policies to improve sample efficiency when learning on these kinds of environments. We also evaluate the impact of learning from human vs simulated experts. Among the different levels of structure that we tested, the graph neural networks (GNNs) show a remarkable superiority by reaching a success rate above 80% with only 50 dialogues when learning from simulated experts. They also show superiority when learning from human experts, although a performance drop was observed. We therefore suggest to concentrate future research efforts on bridging the gap between human data, simulators and automatic evaluators in dialogue frameworks.- Anthology ID:
- 2023.findings-eacl.32
- Volume:
- Findings of the Association for Computational Linguistics: EACL 2023
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 432–441
- Language:
- URL:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2023.findings-eacl.32/
- DOI:
- 10.18653/v1/2023.findings-eacl.32
- Cite (ACL):
- Thibault Cordier, Tanguy Urvoy, Fabrice Lefèvre, and Lina M. Rojas Barahona. 2023. Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues. In Findings of the Association for Computational Linguistics: EACL 2023, pages 432–441, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues (Cordier et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2023.findings-eacl.32.pdf