Learning to Learn to be Right for the Right Reasons
Pride Kavumba, Benjamin Heinzerling, Ana Brassard, Kentaro Inui
Abstract
Improving model generalization on held-out data is one of the core objectives in common- sense reasoning. Recent work has shown that models trained on the dataset with superficial cues tend to perform well on the easy test set with superficial cues but perform poorly on the hard test set without superficial cues. Previous approaches have resorted to manual methods of encouraging models not to overfit to superficial cues. While some of the methods have improved performance on hard instances, they also lead to degraded performance on easy in- stances. Here, we propose to explicitly learn a model that does well on both the easy test set with superficial cues and the hard test set without superficial cues. Using a meta-learning objective, we learn such a model that improves performance on both the easy test set and the hard test set. By evaluating our models on Choice of Plausible Alternatives (COPA) and Commonsense Explanation, we show that our proposed method leads to improved performance on both the easy test set and the hard test set upon which we observe up to 16.5 percentage points improvement over the baseline.- Anthology ID:
- 2021.naacl-main.304
- Volume:
- Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Month:
- June
- Year:
- 2021
- Address:
- Online
- Editors:
- Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3890–3898
- Language:
- URL:
- https://aclanthology.org/2021.naacl-main.304
- DOI:
- 10.18653/v1/2021.naacl-main.304
- Cite (ACL):
- Pride Kavumba, Benjamin Heinzerling, Ana Brassard, and Kentaro Inui. 2021. Learning to Learn to be Right for the Right Reasons. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3890–3898, Online. Association for Computational Linguistics.
- Cite (Informal):
- Learning to Learn to be Right for the Right Reasons (Kavumba et al., NAACL 2021)
- PDF:
- https://preview.aclanthology.org/bionlp-24-ingestion/2021.naacl-main.304.pdf
- Data
- COPA, SuperGLUE