A Report on the FigLang 2024 Shared Task on Multimodal Figurative Language
Shreyas Kulkarni, Arkadiy Saakyan, Tuhin Chakrabarty, Smaranda Muresan
Abstract
We present the outcomes of the Multimodal Figurative Language Shared Task held at the 4th Workshop on Figurative Language Processing (FigLang 2024) co-located at NAACL 2024. The task utilized the V-FLUTE dataset which is comprised of <image, text> pairs that use figurative language and includes detailed textual explanations for the entailment or contradiction relationship of each pair. The challenge for participants was to develop models capable of accurately identifying the visual entailment relationship in these multimodal instances and generating persuasive free-text explanations. The results showed that the participants’ models significantly outperformed the initial baselines in both automated and human evaluations. We also provide an overview of the systems submitted and analyze the results of the evaluations. All participating systems outperformed the LLaVA-ZS baseline, provided by us in F1-score.- Anthology ID:
- 2024.figlang-1.16
- Volume:
- Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico (Hybrid)
- Editors:
- Debanjan Ghosh, Smaranda Muresan, Anna Feldman, Tuhin Chakrabarty, Emmy Liu
- Venues:
- Fig-Lang | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 115–119
- Language:
- URL:
- https://aclanthology.org/2024.figlang-1.16
- DOI:
- 10.18653/v1/2024.figlang-1.16
- Cite (ACL):
- Shreyas Kulkarni, Arkadiy Saakyan, Tuhin Chakrabarty, and Smaranda Muresan. 2024. A Report on the FigLang 2024 Shared Task on Multimodal Figurative Language. In Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024), pages 115–119, Mexico City, Mexico (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- A Report on the FigLang 2024 Shared Task on Multimodal Figurative Language (Kulkarni et al., Fig-Lang-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.figlang-1.16.pdf