Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework
Aaron Steven White, Pushpendre Rastogi, Kevin Duh, Benjamin Van Durme
Abstract
We propose to unify a variety of existing semantic classification tasks, such as semantic role labeling, anaphora resolution, and paraphrase detection, under the heading of Recognizing Textual Entailment (RTE). We present a general strategy to automatically generate one or more sentential hypotheses based on an input sentence and pre-existing manual semantic annotations. The resulting suite of datasets enables us to probe a statistical RTE model’s performance on different aspects of semantics. We demonstrate the value of this approach by investigating the behavior of a popular neural network RTE model.- Anthology ID:
- I17-1100
- Volume:
- Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Month:
- November
- Year:
- 2017
- Address:
- Taipei, Taiwan
- Editors:
- Greg Kondrak, Taro Watanabe
- Venue:
- IJCNLP
- SIG:
- Publisher:
- Asian Federation of Natural Language Processing
- Note:
- Pages:
- 996–1005
- Language:
- URL:
- https://aclanthology.org/I17-1100
- DOI:
- Cite (ACL):
- Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van Durme. 2017. Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 996–1005, Taipei, Taiwan. Asian Federation of Natural Language Processing.
- Cite (Informal):
- Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework (White et al., IJCNLP 2017)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/I17-1100.pdf
- Data
- SNLI