Wiki-VEL: Visual Entity Linking for Structured Data on Wikimedia Commons

Philipp Bielefeld, Jasmin Geppert, Necdet Güven, Melna John, Adrian Ziupka, Lucie-Aimée Kaffee, Russa Biswas, Gerard De Melo

Abstract
Describing Wikimedia Commons images using Wikidata’s structured data enables a wide range of automation tasks, such as search and organization, as well as downstream tasks, such as labeling images or training machine learning models. However, there is currently a lack of structured data-labelled images on Wikimedia Commons.To close this gap, we propose the task of Visual Entity Linking (VEL) for Wikimedia Commons, in which we create new labels for Wikimedia Commons images from Wikidata items. VEL is a crucial tool for improving information retrieval, search, content understanding, cross-modal applications, and various machine-learning tasks. In this paper, we propose a method to create new labels for Wikimedia Commons images from Wikidata items. To this end, we create a novel dataset leveraging community-created structured data on Wikimedia Commons and fine-tuning pre-trained models based on the CLIP architecture. Although the best-performing models show promising results, the study also identifies key challenges of the data and the task.
Anthology ID:
2024.alvr-1.16
Volume:
Proceedings of the 3rd Workshop on Advances in Language and Vision Research (ALVR)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Jing Gu, Tsu-Jui (Ray) Fu, Drew Hudson, Asli Celikyilmaz, William Wang
Venues:
ALVR | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
186–194
Language:
URL:
https://aclanthology.org/2024.alvr-1.16
DOI:
Bibkey:
Cite (ACL):
Philipp Bielefeld, Jasmin Geppert, Necdet Güven, Melna John, Adrian Ziupka, Lucie-Aimée Kaffee, Russa Biswas, and Gerard De Melo. 2024. Wiki-VEL: Visual Entity Linking for Structured Data on Wikimedia Commons. In Proceedings of the 3rd Workshop on Advances in Language and Vision Research (ALVR), pages 186–194, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Wiki-VEL: Visual Entity Linking for Structured Data on Wikimedia Commons (Bielefeld et al., ALVR-WS 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.alvr-1.16.pdf
PDF Search