An RNN-based Binary Classifier for the Story Cloze Test
Melissa Roemmele, Sosuke Kobayashi, Naoya Inoue, Andrew Gordon
Abstract
The Story Cloze Test consists of choosing a sentence that best completes a story given two choices. In this paper we present a system that performs this task using a supervised binary classifier on top of a recurrent neural network to predict the probability that a given story ending is correct. The classifier is trained to distinguish correct story endings given in the training data from incorrect ones that we artificially generate. Our experiments evaluate different methods for generating these negative examples, as well as different embedding-based representations of the stories. Our best result obtains 67.2% accuracy on the test set, outperforming the existing top baseline of 58.5%.- Anthology ID:
- W17-0911
- Volume:
- Proceedings of the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics
- Month:
- April
- Year:
- 2017
- Address:
- Valencia, Spain
- Venue:
- LSDSem
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 74–80
- Language:
- URL:
- https://aclanthology.org/W17-0911
- DOI:
- 10.18653/v1/W17-0911
- Cite (ACL):
- Melissa Roemmele, Sosuke Kobayashi, Naoya Inoue, and Andrew Gordon. 2017. An RNN-based Binary Classifier for the Story Cloze Test. In Proceedings of the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics, pages 74–80, Valencia, Spain. Association for Computational Linguistics.
- Cite (Informal):
- An RNN-based Binary Classifier for the Story Cloze Test (Roemmele et al., LSDSem 2017)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/W17-0911.pdf
- Data
- ROCStories