Abstract
We present an in-depth analysis of metaphor novelty, a relatively overlooked phenomenon in NLP. Novel metaphors have been analyzed via scores derived from crowdsourcing in NLP, while in theoretical work they are often defined by comparison to senses in dictionary entries. We reannotate metaphorically used words in the large VU Amsterdam Metaphor Corpus based on whether their metaphoric meaning is present in the dictionary. Based on this, we find that perceived metaphor novelty often clash with the dictionary based definition. We use the new labels to evaluate the performance of state-of-the-art language models for automatic metaphor detection and notice that novel metaphors according to our dictionary-based definition are easier to identify than novel metaphors according to crowd-sourced novelty scores. In a subsequent analysis, we study the correlation between high novelty scores and word frequencies in the pretraining and finetuning corpora, as well as potential problems with rare words for pre-trained language models. In line with previous works, we find a negative correlation between word frequency in the training data and novelty scores and we link these aspects to problems with the tokenization of BERT and RoBERTa.- Anthology ID:
- 2024.law-1.9
- Volume:
- Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII)
- Month:
- March
- Year:
- 2024
- Address:
- St. Julians, Malta
- Editors:
- Sophie Henning, Manfred Stede
- Venues:
- LAW | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 87–97
- Language:
- URL:
- https://aclanthology.org/2024.law-1.9
- DOI:
- Cite (ACL):
- Sebastian Reimann and Tatjana Scheffler. 2024. When is a Metaphor Actually Novel? Annotating Metaphor Novelty in the Context of Automatic Metaphor Detection. In Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII), pages 87–97, St. Julians, Malta. Association for Computational Linguistics.
- Cite (Informal):
- When is a Metaphor Actually Novel? Annotating Metaphor Novelty in the Context of Automatic Metaphor Detection (Reimann & Scheffler, LAW-WS 2024)
- PDF:
- https://preview.aclanthology.org/landing_page/2024.law-1.9.pdf