DD-TIG at SemEval-2022 Task 5: Investigating the Relationships Between Multimodal and Unimodal Information in Misogynous Memes Detection and Classification

Ziming Zhou, Han Zhao, Jingjing Dong, Ning Ding, Xiaolong Liu, Kangli Zhang

Abstract
This paper describes our submission for task 5 Multimedia Automatic Misogyny Identification (MAMI) at SemEval-2022. The task is designed to detect and classify misogynous memes. To utilize both textual and visual information presented in a meme, we investigate several of the most recent visual language transformer-based multimodal models and choose ERNIE-ViL-Large as our base model. For subtask A, with observations of models’ overfitting on unimodal patterns, strategies are proposed to mitigate problems of biased words and template memes. For subtask B, we transform this multi-label problem into a multi-class one and experiment with oversampling and complementary techniques. Our approach places 2nd for subtask A and 5th for subtask B in this competition.
Anthology ID:
2022.semeval-1.77
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
SemEval
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
563–570
Language:
URL:
https://aclanthology.org/2022.semeval-1.77
DOI:
10.18653/v1/2022.semeval-1.77
Bibkey:
Cite (ACL):
Ziming Zhou, Han Zhao, Jingjing Dong, Ning Ding, Xiaolong Liu, and Kangli Zhang. 2022. DD-TIG at SemEval-2022 Task 5: Investigating the Relationships Between Multimodal and Unimodal Information in Misogynous Memes Detection and Classification. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 563–570, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
DD-TIG at SemEval-2022 Task 5: Investigating the Relationships Between Multimodal and Unimodal Information in Misogynous Memes Detection and Classification (Zhou et al., SemEval 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.semeval-1.77.pdf
Data
Hateful Memes
PDF Search