Abstract
Misogynistic memes are rampant on social media, and often convey their messages using multimodal signals (e.g., images paired with derogatory text or captions). However, to date very few multimodal systems have been leveraged for the detection of misogynistic memes. Recently, researchers have turned to contrastive learning, and most notably OpenAI’s CLIP model, is an innovative solution to a variety of multimodal tasks. In this work, we experiment with contrastive learning to address the detection of misogynistic memes within the context of SemEval 2022 Task 5. Although our model does not achieve top results, these experiments provide important exploratory findings for this task. We conduct a detailed error analysis, revealing promising clues and offering a foundation for follow-up work.- Anthology ID:
- 2022.semeval-1.109
- Volume:
- Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Editors:
- Guy Emerson, Natalie Schluter, Gabriel Stanovsky, Ritesh Kumar, Alexis Palmer, Nathan Schneider, Siddharth Singh, Shyam Ratan
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 785–792
- Language:
- URL:
- https://aclanthology.org/2022.semeval-1.109
- DOI:
- 10.18653/v1/2022.semeval-1.109
- Cite (ACL):
- Charic Farinango Cuervo and Natalie Parde. 2022. Exploring Contrastive Learning for Multimodal Detection of Misogynistic Memes. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 785–792, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- Exploring Contrastive Learning for Multimodal Detection of Misogynistic Memes (Farinango Cuervo & Parde, SemEval 2022)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2022.semeval-1.109.pdf
- Code
- moein-shariatnia/OpenAI-CLIP + additional community code
- Data
- Hateful Memes