IIITH at SemEval-2022 Task 5: A comparative study of deep learning models for identifying misogynous memes

Tathagata Raha, Sagar Joshi, Vasudeva Varma


Abstract
This paper provides a comparison of different deep learning methods for identifying misogynous memes for SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification. In this task, we experiment with architectures in the identification of misogynous content in memes by making use of text and image-based information. The different deep learning methods compared in this paper are: (i) unimodal image or text models (ii) fusion of unimodal models (iii) multimodal transformers models and (iv) transformers further pretrained on a multimodal task. From our experiments, we found pretrained multimodal transformer architectures to strongly outperform the models involving the fusion of representation from both the modalities.
Anthology ID:
2022.semeval-1.92
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
SemEval
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
673–678
Language:
URL:
https://aclanthology.org/2022.semeval-1.92
DOI:
10.18653/v1/2022.semeval-1.92
Bibkey:
Cite (ACL):
Tathagata Raha, Sagar Joshi, and Vasudeva Varma. 2022. IIITH at SemEval-2022 Task 5: A comparative study of deep learning models for identifying misogynous memes. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 673–678, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
IIITH at SemEval-2022 Task 5: A comparative study of deep learning models for identifying misogynous memes (Raha et al., SemEval 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/2022.semeval-1.92.pdf
Data
COCOHateful MemesVisual Genome