Pragmatic and Logical Inferences in NLI Systems: The Case of Conjunction Buttressing
Paolo Pedinotti, Emmanuele Chersoni, Enrico Santus, Alessandro Lenci
Abstract
An intelligent system is expected to perform reasonable inferences, accounting for both the literal meaning of a word and the meanings a word can acquire in different contexts. A specific kind of inference concerns the connective and, which in some cases gives rise to a temporal succession or causal interpretation in contrast with the logic, commutative one (Levinson, 2000). In this work, we investigate the phenomenon by creating a new dataset for evaluating the interpretation of and by NLI systems, which we use to test three Transformer-based models. Our results show that all systems generalize patterns that are consistent with both the logical and the pragmatic interpretation, perform inferences that are inconsistent with each other, and show clear divergences with both theoretical accounts and humans’ behavior.- Anthology ID:
- 2022.unimplicit-1.2
- Volume:
- Proceedings of the Second Workshop on Understanding Implicit and Underspecified Language
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, USA
- Editors:
- Valentina Pyatkin, Daniel Fried, Talita Anthonio
- Venue:
- unimplicit
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8–16
- Language:
- URL:
- https://aclanthology.org/2022.unimplicit-1.2
- DOI:
- 10.18653/v1/2022.unimplicit-1.2
- Cite (ACL):
- Paolo Pedinotti, Emmanuele Chersoni, Enrico Santus, and Alessandro Lenci. 2022. Pragmatic and Logical Inferences in NLI Systems: The Case of Conjunction Buttressing. In Proceedings of the Second Workshop on Understanding Implicit and Underspecified Language, pages 8–16, Seattle, USA. Association for Computational Linguistics.
- Cite (Informal):
- Pragmatic and Logical Inferences in NLI Systems: The Case of Conjunction Buttressing (Pedinotti et al., unimplicit 2022)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2022.unimplicit-1.2.pdf
- Data
- MultiNLI