JNLP at SemEval-2025 Task 1: Multimodal Idiomaticity Representation with Large Language Models

Blake Matheny, Phuong Nguyen, Minh Nguyen


Abstract
Idioms and figurative language are nuanced linguistic phenomena that transport semanticity and culture via non-compositional multi-word expressions. This type of figurative language remains difficult for small and large language models to handle. Various attempts have been made to identify idiomaticity in text. The approach presented in this paper represents an intuitive attempt to accurately address Task 1: AdMIRe Subtask A to correctly order a series of images and captions by concatenating the image captions as a sequence. The methods employ the reliability of a pre-trained vision and language model for the image-type task and a large language model with instruction fine-tuning for a causal language model approach to handle the caption portion of the task. The results are informative for future iterations, but not comparably substantial.
Anthology ID:
2025.semeval-1.195
Volume:
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Sara Rosenthal, Aiala Rosá, Debanjan Ghosh, Marcos Zampieri
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1479–1484
Language:
URL:
https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.195/
DOI:
Bibkey:
Cite (ACL):
Blake Matheny, Phuong Nguyen, and Minh Nguyen. 2025. JNLP at SemEval-2025 Task 1: Multimodal Idiomaticity Representation with Large Language Models. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 1479–1484, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
JNLP at SemEval-2025 Task 1: Multimodal Idiomaticity Representation with Large Language Models (Matheny et al., SemEval 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.195.pdf