Penguins Don’t Fly: Reasoning about Generics through Instantiations and Exceptions

Emily Allaway, Jena D. Hwang, Chandra Bhagavatula, Kathleen McKeown, Doug Downey, Yejin Choi


Abstract
Generics express generalizations about the world (e.g., birds can fly) that are not universally true (e.g., newborn birds and penguins cannot fly). Commonsense knowledge bases, used extensively in NLP, encode some generic knowledge but rarely enumerate such exceptions and knowing when a generic statement holds or does not hold true is crucial for developing a comprehensive understanding of generics. We present a novel framework informed by linguistic theory to generate exemplars—specific cases when a generic holds true or false. We generate ~19k exemplars for ~650 generics and show that our framework outperforms a strong GPT-3 baseline by 12.8 precision points. Our analysis highlights the importance of linguistic theory-based controllability for generating exemplars, the insufficiency of knowledge bases as a source of exemplars, and the challenges exemplars pose for the task of natural language inference.
Anthology ID:
2023.eacl-main.192
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2618–2635
Language:
URL:
https://aclanthology.org/2023.eacl-main.192
DOI:
10.18653/v1/2023.eacl-main.192
Bibkey:
Cite (ACL):
Emily Allaway, Jena D. Hwang, Chandra Bhagavatula, Kathleen McKeown, Doug Downey, and Yejin Choi. 2023. Penguins Don’t Fly: Reasoning about Generics through Instantiations and Exceptions. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2618–2635, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Penguins Don’t Fly: Reasoning about Generics through Instantiations and Exceptions (Allaway et al., EACL 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-1/2023.eacl-main.192.pdf
Video:
 https://preview.aclanthology.org/nschneid-patch-1/2023.eacl-main.192.mp4