@inproceedings{rajkumar-etal-2026-multimodal,
    title = "Multimodal Transformer Framework for Multilingual Harmful Meme Classification",
    author = "Rajkumar, Charmathi  and
      Subramanian, Malliga  and
      Chakravarthi, Bharathi Raja",
    editor = "Chakravarthi, Bharathi Raja  and
      B, Bharathi  and
      Buitelaar, Paul  and
      Thenmozhi, Durairaj  and
      Garc{\'i}a Cumbreras, Miguel {\'A}ngel  and
      Jim{\'e}nez Zafra, Salud Mar{\'i}a",
    booktitle = "Proceedings of the Sixth Workshop on Language Technology for Equality, Diversity, Inclusion",
    month = jul,
    year = "2026",
    address = "Virtual (Online)",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.ltedi-1.9/",
    pages = "99--107",
    ISBN = "979-8-89176-424-8",
    abstract = "The rapid expansion of social media platforms has led to a significant increase in the spread of harmful content, including misogynistic, homophobic, and transphobic memes. Detecting such content is challenging because memes often combine textual and visual elements and frequently appear in multilingual and culturally diverse contexts. This study proposes a multimodal transformer-based framework for multilingual harmful meme classification that integrates textual and visual representations to improve detection performance. The proposed architecture employs XLM-RoBERTa for multilingual text encoding and the Swin Transformer for hierarchical visual feature extraction. A cross-attention fusion mechanism is introduced to enable meaningful interaction between textual and visual modalities. The fused representation is then processed through a classification layer to perform multi-class prediction. Experiments are conducted across multiple datasets covering eight languages and three harmful content categories: misogyny, homophobia/transphobia, and hate speech. The model is evaluated using the macro-F1 score and demonstrates consistent improvements over baseline multimodal systems across both high-resource and low-resource languages. The results highlight the effectiveness of transformer-based multimodal architectures in capturing implicit and contextual harmful signals present in memes. The study contributes to the development of robust multilingual systems for harmful content detection and supports efforts toward creating safer and more inclusive online environments."
}