NOPE: A Corpus of Naturally-Occurring Presuppositions in English

Alicia Parrish, Sebastian Schuster, Alex Warstadt, Omar Agha, Soo-Hwan Lee, Zhuoye Zhao, Samuel R. Bowman, Tal Linzen


Abstract
Understanding language requires grasping not only the overtly stated content, but also making inferences about things that were left unsaid. These inferences include presuppositions, a phenomenon by which a listener learns about new information through reasoning about what a speaker takes as given. Presuppositions require complex understanding of the lexical and syntactic properties that trigger them as well as the broader conversational context. In this work, we introduce the Naturally-Occurring Presuppositions in English (NOPE) Corpus to investigate the context-sensitivity of 10 different types of presupposition triggers and to evaluate machine learning models’ ability to predict human inferences. We find that most of the triggers we investigate exhibit moderate variability. We further find that transformer-based models draw correct inferences in simple cases involving presuppositions, but they fail to capture the minority of exceptional cases in which human judgments reveal complex interactions between context and triggers.
Anthology ID:
2021.conll-1.28
Volume:
Proceedings of the 25th Conference on Computational Natural Language Learning
Month:
November
Year:
2021
Address:
Online
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
349–366
Language:
URL:
https://aclanthology.org/2021.conll-1.28
DOI:
10.18653/v1/2021.conll-1.28
Bibkey:
Cite (ACL):
Alicia Parrish, Sebastian Schuster, Alex Warstadt, Omar Agha, Soo-Hwan Lee, Zhuoye Zhao, Samuel R. Bowman, and Tal Linzen. 2021. NOPE: A Corpus of Naturally-Occurring Presuppositions in English. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 349–366, Online. Association for Computational Linguistics.
Cite (Informal):
NOPE: A Corpus of Naturally-Occurring Presuppositions in English (Parrish et al., CoNLL 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.conll-1.28.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2021.conll-1.28.mp4
Code
 nyu-mll/nope
Data
ANLIMultiNLISNLI