@inproceedings{orny-etal-2026-team,
    title = "Team Oryu@{CH}i{PSAL} 2026: Integrating Text and Vision Transformers for Multimodal Hate Speech Detection in Memes",
    author = "Orny, Noore Tamanna  and
      Moni, Joyeta Barua  and
      Kabir, Md. Abtahee  and
      Murad, Hasan",
    editor = "Sarveswaran, Kengatharaiyer  and
      Vaidya, Ashwini",
    booktitle = "Proceedings of the Second workshop on Challenges in Processing {S}outh {A}sian Languages ({CH}i{PSAL}2026)",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/ingest-nlpsi/2026.chipsal-1.29/",
    doi = "10.63317/2fb8jq6jj36y",
    pages = "284--291",
    abstract = "With the proliferation of multimodal content on various social media platforms, automated hate speech detection has emerged as a challenge, especially in meme-based communication, where meaning arises from interactions between text and images. In these situations, unimodal techniques are inadequate in capturing semantics. In order to address such issues, a late-fusion-based multimodal hate speech detection framework has been proposed and implemented for the CHiPSAL shared task. In the proposed framework, multimodal content is processed by utilizing XLM-RoBERTa for multilingual text representation and a Vision Transformer (ViT) for visual representation. Both modal representations are fused using a fully connected classification head and are used for binary hate speech detection. The findings suggest that multimodal content effectively captures features from individual modalities and helps improve hate speech detection accuracy by obtaining a Macro F1-score of 0.66 and ranking 5th on the leaderboard. Also, transformer-based multimodal fusion performs effectively and acts as a reliable baseline for hate speech detection in low-resource multilingual meme-based communication scenarios."
}