Logically at the Constraint 2022: Multimodal role labelling

Ludovic Kun, Jayesh Bankoti, David Kiskovski


Abstract
This paper describes our system for the Constraint 2022 challenge at ACL 2022, whose goal is to detect which entities are glorified, vilified or victimised, within a meme . The task should be done considering the perspective of the meme’s author. In our work, the challenge is treated as a multi-class classification task. For a given pair of a meme and an entity, we need to classify whether the entity is being referenced as Hero, a Villain, a Victim or Other. Our solution combines (ensembling) different models based on Unimodal (Text only) model and Multimodal model (Text + Images). We conduct several experiments and benchmarks different competitive pre-trained transformers and vision models in this work. Our solution, based on an ensembling method, is ranked first on the leaderboard and obtains a macro F1-score of 0.58 on test set. The code for the experiments and results are available at https://bitbucket.org/logicallydevs/constraint_2022/src/master/
Anthology ID:
2022.constraint-1.4
Volume:
Proceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venue:
CONSTRAINT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
24–34
Language:
URL:
https://aclanthology.org/2022.constraint-1.4
DOI:
10.18653/v1/2022.constraint-1.4
Bibkey:
Cite (ACL):
Ludovic Kun, Jayesh Bankoti, and David Kiskovski. 2022. Logically at the Constraint 2022: Multimodal role labelling. In Proceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations, pages 24–34, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Logically at the Constraint 2022: Multimodal role labelling (Kun et al., CONSTRAINT 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.constraint-1.4.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2022.constraint-1.4.mp4