Abstract
Algorithms for text-generation in dialogue can be misguided. For example, in task-oriented settings, reinforcement learning that optimizes only task-success can lead to abysmal lexical diversity. We hypothesize this is due to poor theoretical understanding of the objectives in text-generation and their relation to the learning process (i.e., model training). To this end, we propose a new theoretical framework for learning to generate text in dialogue. Compared to existing theories of learning, our framework allows for analysis of the multi-faceted goals inherent to text-generation. We use our framework to develop theoretical guarantees for learners that adapt to unseen data. As an example, we apply our theory to study data-shift within a cooperative learning algorithm proposed for the GuessWhat?! visual dialogue game. From this insight, we propose a new algorithm, and empirically, we demonstrate our proposal improves both task-success and human-likeness of the generated text. Finally, we show statistics from our theory are empirically predictive of multiple qualities of the generated dialogue, suggesting our theory is useful for model-selection when human evaluations are not available.- Anthology ID:
- 2022.findings-aacl.4
- Volume:
- Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022
- Month:
- November
- Year:
- 2022
- Address:
- Online only
- Editors:
- Yulan He, Heng Ji, Sujian Li, Yang Liu, Chua-Hui Chang
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 30–53
- Language:
- URL:
- https://aclanthology.org/2022.findings-aacl.4
- DOI:
- Cite (ACL):
- Anthony Sicilia and Malihe Alikhani. 2022. LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue. In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pages 30–53, Online only. Association for Computational Linguistics.
- Cite (Informal):
- LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue (Sicilia & Alikhani, Findings 2022)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2022.findings-aacl.4.pdf