Abstract
Figurative language is widespread in human language (Lakoff and Johnson, 2008) posing potential challenges in NLP applications. In this paper, we investigate the effect of figurative language on the task of question answering (QA). We construct FigQA, a test set of 400 yes-no questions with figurative and non-figurative contexts, extracted from product reviews and restaurant reviews. We demonstrate that a state-of-the-art RoBERTa QA model has considerably lower performance in question answering when the contexts are figurative rather than literal, indicating a gap in current models. We propose a general method for improving the performance of QA models by converting the figurative contexts into non-figurative by prompting GPT-3, and demonstrate its effectiveness. Our results indicate a need for building QA models infused with figurative language understanding capabilities.- Anthology ID:
- 2022.flp-1.23
- Volume:
- Proceedings of the 3rd Workshop on Figurative Language Processing (FLP)
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates (Hybrid)
- Editors:
- Debanjan Ghosh, Beata Beigman Klebanov, Smaranda Muresan, Anna Feldman, Soujanya Poria, Tuhin Chakrabarty
- Venue:
- Fig-Lang
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 160–166
- Language:
- URL:
- https://aclanthology.org/2022.flp-1.23
- DOI:
- 10.18653/v1/2022.flp-1.23
- Cite (ACL):
- Geetanjali Rakshit and Jeffrey Flanigan. 2022. FigurativeQA: A Test Benchmark for Figurativeness Comprehension for Question Answering. In Proceedings of the 3rd Workshop on Figurative Language Processing (FLP), pages 160–166, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- FigurativeQA: A Test Benchmark for Figurativeness Comprehension for Question Answering (Rakshit & Flanigan, Fig-Lang 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2022.flp-1.23.pdf