Abstract
We consider the ROC story cloze task (Mostafazadeh et al., 2016) and present several findings. We develop a model that uses hierarchical recurrent networks with attention to encode the sentences in the story and score candidate endings. By discarding the large training set and only training on the validation set, we achieve an accuracy of 74.7%. Even when we discard the story plots (sentences before the ending) and only train to choose the better of two endings, we can still reach 72.5%. We then analyze this “ending-only” task setting. We estimate human accuracy to be 78% and find several types of clues that lead to this high accuracy, including those related to sentiment, negation, and general ending likelihood regardless of the story context.- Anthology ID:
- P17-2097
- Volume:
- Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
- Month:
- July
- Year:
- 2017
- Address:
- Vancouver, Canada
- Editors:
- Regina Barzilay, Min-Yen Kan
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 616–622
- Language:
- URL:
- https://aclanthology.org/P17-2097
- DOI:
- 10.18653/v1/P17-2097
- Cite (ACL):
- Zheng Cai, Lifu Tu, and Kevin Gimpel. 2017. Pay Attention to the Ending:Strong Neural Baselines for the ROC Story Cloze Task. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 616–622, Vancouver, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Pay Attention to the Ending:Strong Neural Baselines for the ROC Story Cloze Task (Cai et al., ACL 2017)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/P17-2097.pdf