Augmenting Neural Response Generation with Context-Aware Topical Attention

Nouha Dziri, Ehsan Kamalloo, Kory Mathewson, Osmar Zaiane

Abstract
Sequence-to-Sequence (Seq2Seq) models have witnessed a notable success in generating natural conversational exchanges. Notwithstanding the syntactically well-formed responses generated by these neural network models, they are prone to be acontextual, short and generic. In this work, we introduce a Topical Hierarchical Recurrent Encoder Decoder (THRED), a novel, fully data-driven, multi-turn response generation system intended to produce contextual and topic-aware responses. Our model is built upon the basic Seq2Seq model by augmenting it with a hierarchical joint attention mechanism that incorporates topical concepts and previous interactions into the response generation. To train our model, we provide a clean and high-quality conversational dataset mined from Reddit comments. We evaluate THRED on two novel automated metrics, dubbed Semantic Similarity and Response Echo Index, as well as with human evaluation. Our experiments demonstrate that the proposed model is able to generate more diverse and contextually relevant responses compared to the strong baselines.
Anthology ID:
W19-4103
Volume:
Proceedings of the First Workshop on NLP for Conversational AI
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Yun-Nung Chen, Tania Bedrax-Weiss, Dilek Hakkani-Tur, Anuj Kumar, Mike Lewis, Thang-Minh Luong, Pei-Hao Su, Tsung-Hsien Wen
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
18–31
Language:
URL:
https://aclanthology.org/W19-4103
DOI:
10.18653/v1/W19-4103
Bibkey:
Cite (ACL):
Nouha Dziri, Ehsan Kamalloo, Kory Mathewson, and Osmar Zaiane. 2019. Augmenting Neural Response Generation with Context-Aware Topical Attention. In Proceedings of the First Workshop on NLP for Conversational AI, pages 18–31, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Augmenting Neural Response Generation with Context-Aware Topical Attention (Dziri et al., ACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp-22-attachments/W19-4103.pdf
Code
 nouhadziri/THRED
Data
Reddit Conversation Corpus
PDF Search Code