Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction

Hirokazu Kiyomaru, Kazumasa Omura, Yugo Murawaki, Daisuke Kawahara, Sadao Kurohashi

[How to correct problems with metadata yourself]


Abstract
Typical event sequences are an important class of commonsense knowledge. Formalizing the task as the generation of a next event conditioned on a current event, previous work in event prediction employs sequence-to-sequence (seq2seq) models. However, what can happen after a given event is usually diverse, a fact that can hardly be captured by deterministic models. In this paper, we propose to incorporate a conditional variational autoencoder (CVAE) into seq2seq for its ability to represent diverse next events as a probabilistic distribution. We further extend the CVAE-based seq2seq with a reconstruction mechanism to prevent the model from concentrating on highly typical events. To facilitate fair and systematic evaluation of the diversity-aware models, we also extend existing evaluation datasets by tying each current event to multiple next events. Experiments show that the CVAE-based models drastically outperform deterministic models in terms of precision and that the reconstruction mechanism improves the recall of CVAE-based models without sacrificing precision.
Anthology ID:
D19-6014
Volume:
Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Simon Ostermann, Sheng Zhang, Michael Roth, Peter Clark
Venue:
WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
113–122
Language:
URL:
https://aclanthology.org/D19-6014
DOI:
10.18653/v1/D19-6014
Bibkey:
Cite (ACL):
Hirokazu Kiyomaru, Kazumasa Omura, Yugo Murawaki, Daisuke Kawahara, and Sadao Kurohashi. 2019. Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction. In Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing, pages 113–122, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction (Kiyomaru et al., 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/teach-a-man-to-fish/D19-6014.pdf