NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation

Xiaoyu Shen, Hui Su, Wenjie Li, Dietrich Klakow


Abstract
Sequence-to-Sequence (seq2seq) models have become overwhelmingly popular in building end-to-end trainable dialogue systems. Though highly efficient in learning the backbone of human-computer communications, they suffer from the problem of strongly favoring short generic responses. In this paper, we argue that a good response should smoothly connect both the preceding dialogue history and the following conversations. We strengthen this connection by mutual information maximization. To sidestep the non-differentiability of discrete natural language tokens, we introduce an auxiliary continuous code space and map such code space to a learnable prior distribution for generation purpose. Experiments on two dialogue datasets validate the effectiveness of our model, where the generated responses are closely related to the dialogue context and lead to more interactive conversations.
Anthology ID:
D18-1463
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
4316–4327
Language:
URL:
https://aclanthology.org/D18-1463
DOI:
10.18653/v1/D18-1463
Bibkey:
Cite (ACL):
Xiaoyu Shen, Hui Su, Wenjie Li, and Dietrich Klakow. 2018. NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4316–4327, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation (Shen et al., EMNLP 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/starsem-semeval-split/D18-1463.pdf
Attachment:
 D18-1463.Attachment.pdf
Video:
 https://vimeo.com/306163081