A howling success or a working sea? Testing what BERT knows about metaphors
Paolo Pedinotti, Eliana Di Palma, Ludovica Cerini, Alessandro Lenci
Abstract
Metaphor is a widespread linguistic and cognitive phenomenon that is ruled by mechanisms which have received attention in the literature. Transformer Language Models such as BERT have brought improvements in metaphor-related tasks. However, they have been used only in application contexts, while their knowledge of the phenomenon has not been analyzed. To test what BERT knows about metaphors, we challenge it on a new dataset that we designed to test various aspects of this phenomenon such as variations in linguistic structure, variations in conventionality, the boundaries of the plausibility of a metaphor and the interpretations that we attribute to metaphoric expressions. Results bring out some tendencies that suggest that the model can reproduce some human intuitions about metaphors.- Anthology ID:
- 2021.blackboxnlp-1.13
- Volume:
- Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Editors:
- Jasmijn Bastings, Yonatan Belinkov, Emmanuel Dupoux, Mario Giulianelli, Dieuwke Hupkes, Yuval Pinter, Hassan Sajjad
- Venue:
- BlackboxNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 192–204
- Language:
- URL:
- https://aclanthology.org/2021.blackboxnlp-1.13
- DOI:
- 10.18653/v1/2021.blackboxnlp-1.13
- Cite (ACL):
- Paolo Pedinotti, Eliana Di Palma, Ludovica Cerini, and Alessandro Lenci. 2021. A howling success or a working sea? Testing what BERT knows about metaphors. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 192–204, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- A howling success or a working sea? Testing what BERT knows about metaphors (Pedinotti et al., BlackboxNLP 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2021.blackboxnlp-1.13.pdf