Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction
Hirokazu Kiyomaru, Kazumasa Omura, Yugo Murawaki, Daisuke Kawahara, Sadao Kurohashi
Abstract
Typical event sequences are an important class of commonsense knowledge. Formalizing the task as the generation of a next event conditioned on a current event, previous work in event prediction employs sequence-to-sequence (seq2seq) models. However, what can happen after a given event is usually diverse, a fact that can hardly be captured by deterministic models. In this paper, we propose to incorporate a conditional variational autoencoder (CVAE) into seq2seq for its ability to represent diverse next events as a probabilistic distribution. We further extend the CVAE-based seq2seq with a reconstruction mechanism to prevent the model from concentrating on highly typical events. To facilitate fair and systematic evaluation of the diversity-aware models, we also extend existing evaluation datasets by tying each current event to multiple next events. Experiments show that the CVAE-based models drastically outperform deterministic models in terms of precision and that the reconstruction mechanism improves the recall of CVAE-based models without sacrificing precision.- Anthology ID:
- D19-6014
- Volume:
- Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Simon Ostermann, Sheng Zhang, Michael Roth, Peter Clark
- Venue:
- WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 113–122
- Language:
- URL:
- https://aclanthology.org/D19-6014
- DOI:
- 10.18653/v1/D19-6014
- Cite (ACL):
- Hirokazu Kiyomaru, Kazumasa Omura, Yugo Murawaki, Daisuke Kawahara, and Sadao Kurohashi. 2019. Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction. In Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing, pages 113–122, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction (Kiyomaru et al., 2019)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/D19-6014.pdf