ML Mob at SemEval-2023 Task 1: Probing CLIP on Visual Word-Sense Disambiguation
Clifton Poth, Martin Hentschel, Tobias Werner, Hannah Sterz, Leonard Bongard
Abstract
Successful word sense disambiguation (WSD)is a fundamental element of natural languageunderstanding. As part of SemEval-2023 Task1, we investigate WSD in a multimodal setting,where ambiguous words are to be matched withcandidate images representing word senses. Wecompare multiple systems based on pre-trainedCLIP models. In our experiments, we findCLIP to have solid zero-shot performance onmonolingual and multilingual data. By em-ploying different fine-tuning techniques, we areable to further enhance performance. However,transferring knowledge between data distribu-tions proves to be more challenging.- Anthology ID:
- 2023.semeval-1.201
- Volume:
- Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1463–1469
- Language:
- URL:
- https://aclanthology.org/2023.semeval-1.201
- DOI:
- 10.18653/v1/2023.semeval-1.201
- Cite (ACL):
- Clifton Poth, Martin Hentschel, Tobias Werner, Hannah Sterz, and Leonard Bongard. 2023. ML Mob at SemEval-2023 Task 1: Probing CLIP on Visual Word-Sense Disambiguation. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1463–1469, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- ML Mob at SemEval-2023 Task 1: Probing CLIP on Visual Word-Sense Disambiguation (Poth et al., SemEval 2023)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2023.semeval-1.201.pdf