A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge
Ali Emami, Adam Trischler, Kaheer Suleman, Jackie Chi Kit Cheung
Abstract
We introduce an automatic system that performs well on two common-sense reasoning tasks, the Winograd Schema Challenge (WSC) and the Choice of Plausible Alternatives (COPA). Problem instances from these tasks require diverse, complex forms of inference and knowledge to solve. Our method uses a knowledge-hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine. It extracts and classifies knowledge from the returned results and weighs it to make a resolution. Our approach improves F1 performance on the WSC by 0.16 over the previous best and is competitive with the state-of-the-art on COPA, demonstrating its general applicability.- Anthology ID:
- N18-4004
- Volume:
- Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop
- Month:
- June
- Year:
- 2018
- Address:
- New Orleans, Louisiana, USA
- Editors:
- Silvio Ricardo Cordeiro, Shereen Oraby, Umashanthi Pavalanathan, Kyeongmin Rim
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 25–31
- Language:
- URL:
- https://aclanthology.org/N18-4004
- DOI:
- 10.18653/v1/N18-4004
- Cite (ACL):
- Ali Emami, Adam Trischler, Kaheer Suleman, and Jackie Chi Kit Cheung. 2018. A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 25–31, New Orleans, Louisiana, USA. Association for Computational Linguistics.
- Cite (Informal):
- A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge (Emami et al., NAACL 2018)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/N18-4004.pdf
- Data
- WSC