JRLV at SemEval-2022 Task 5: The Importance of Visual Elements for Misogyny Identification in Memes

Jason Ravagli, Lorenzo Vaiani


Abstract
Gender discrimination is a serious and widespread problem on social media and online in general. Besides offensive messages, memes are one of the main means of dissemination for such content. With these premises, the MAMI task was proposed at the SemEval-2022, which consists of identifying memes with misogynous characteristics. In this work, we propose a solution to this problem based on Mask R-CNN and VisualBERT that leverages the multimodal nature of the task. Our study focuses on observing how the two sources of data in memes (text and image) and their possible combinations impact performances. Our best result slightly exceeds the higher baseline, but the experiments allowed us to draw important considerations regarding the importance of correctly exploiting the visual information and the relevance of the elements present in the memes images.
Anthology ID:
2022.semeval-1.84
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
SemEval
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
610–617
Language:
URL:
https://aclanthology.org/2022.semeval-1.84
DOI:
10.18653/v1/2022.semeval-1.84
Bibkey:
Cite (ACL):
Jason Ravagli and Lorenzo Vaiani. 2022. JRLV at SemEval-2022 Task 5: The Importance of Visual Elements for Misogyny Identification in Memes. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 610–617, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
JRLV at SemEval-2022 Task 5: The Importance of Visual Elements for Misogyny Identification in Memes (Ravagli & Vaiani, SemEval 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/2022.semeval-1.84.pdf
Video:
 https://preview.aclanthology.org/auto-file-uploads/2022.semeval-1.84.mp4
Data
COCOHateful MemesLVIS