A Report on the FigLang 2024 Shared Task on Multimodal Figurative Language

Shreyas Kulkarni, Arkadiy Saakyan, Tuhin Chakrabarty, Smaranda Muresan

Abstract
We present the outcomes of the Multimodal Figurative Language Shared Task held at the 4th Workshop on Figurative Language Processing (FigLang 2024) co-located at NAACL 2024. The task utilized the V-FLUTE dataset which is comprised of <image, text> pairs that use figurative language and includes detailed textual explanations for the entailment or contradiction relationship of each pair. The challenge for participants was to develop models capable of accurately identifying the visual entailment relationship in these multimodal instances and generating persuasive free-text explanations. The results showed that the participants’ models significantly outperformed the initial baselines in both automated and human evaluations. We also provide an overview of the systems submitted and analyze the results of the evaluations. All participating systems outperformed the LLaVA-ZS baseline, provided by us in F1-score.
Anthology ID:
2024.figlang-1.16
Volume:
Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico (Hybrid)
Editors:
Debanjan Ghosh, Smaranda Muresan, Anna Feldman, Tuhin Chakrabarty, Emmy Liu
Venues:
Fig-Lang | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
115–119
Language:
URL:
https://aclanthology.org/2024.figlang-1.16
DOI:
Bibkey:
Cite (ACL):
Shreyas Kulkarni, Arkadiy Saakyan, Tuhin Chakrabarty, and Smaranda Muresan. 2024. A Report on the FigLang 2024 Shared Task on Multimodal Figurative Language. In Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024), pages 115–119, Mexico City, Mexico (Hybrid). Association for Computational Linguistics.
Cite (Informal):
A Report on the FigLang 2024 Shared Task on Multimodal Figurative Language (Kulkarni et al., Fig-Lang-WS 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.figlang-1.16.pdf
PDF Search