Abstract
This case study investigates the extent to which a language model (GPT-2) is able to capture native speakers’ intuitions about implicit causality in a sentence completion task. Study 1 reproduces earlier results (showing that the model’s surprisal values correlate with the implicit causality bias of the verb; Davis and van Schijndel 2021), and then examine the effects of gender and verb frequency on model performance. Study 2 examines the reasoning ability of GPT-2: Is the model able to produce more sensible motivations for why the subject VERBed the object if the verbs have stronger causality biases? For this study we took care to avoid human raters being biased by obscenities and disfluencies generated by the model.- Anthology ID:
- 2023.iwcs-1.7
- Volume:
- Proceedings of the 15th International Conference on Computational Semantics
- Month:
- June
- Year:
- 2023
- Address:
- Nancy, France
- Editors:
- Maxime Amblard, Ellen Breitholtz
- Venue:
- IWCS
- SIG:
- SIGSEM
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 67–77
- Language:
- URL:
- https://aclanthology.org/2023.iwcs-1.7
- DOI:
- Cite (ACL):
- Minh Hien Huynh, Tomas Lentz, and Emiel van Miltenburg. 2023. Implicit causality in GPT-2: a case study. In Proceedings of the 15th International Conference on Computational Semantics, pages 67–77, Nancy, France. Association for Computational Linguistics.
- Cite (Informal):
- Implicit causality in GPT-2: a case study (Huynh et al., IWCS 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.iwcs-1.7.pdf