Incremental Transformer with Deliberation Decoder for Document Grounded Conversations
Zekang Li, Cheng Niu, Fandong Meng, Yang Feng, Qian Li, Jie Zhou
Abstract
Document Grounded Conversations is a task to generate dialogue responses when chatting about the content of a given document. Obviously, document knowledge plays a critical role in Document Grounded Conversations, while existing dialogue models do not exploit this kind of knowledge effectively enough. In this paper, we propose a novel Transformer-based architecture for multi-turn document grounded conversations. In particular, we devise an Incremental Transformer to encode multi-turn utterances along with knowledge in related documents. Motivated by the human cognitive process, we design a two-pass decoder (Deliberation Decoder) to improve context coherence and knowledge correctness. Our empirical study on a real-world Document Grounded Dataset proves that responses generated by our model significantly outperform competitive baselines on both context coherence and knowledge relevance.- Anthology ID:
- P19-1002
- Volume:
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Anna Korhonen, David Traum, Lluís Màrquez
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 12–21
- Language:
- URL:
- https://aclanthology.org/P19-1002
- DOI:
- 10.18653/v1/P19-1002
- Cite (ACL):
- Zekang Li, Cheng Niu, Fandong Meng, Yang Feng, Qian Li, and Jie Zhou. 2019. Incremental Transformer with Deliberation Decoder for Document Grounded Conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 12–21, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Incremental Transformer with Deliberation Decoder for Document Grounded Conversations (Li et al., ACL 2019)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/P19-1002.pdf
- Code
- lizekang/ITDD + additional community code
- Data
- CMU DoG