@inproceedings{wagle-etal-2026-meme,
    title = "{MEME}-Fusion@{CH}i{PSAL} 2026: Multimodal Ablation Study of Hate Detection and Sentiment Analysis on {N}epali Memes",
    author = "Wagle, Samir  and
      Khanal, Reewaj  and
      Adhikari, Abiral",
    editor = "Sarveswaran, Kengatharaiyer  and
      Vaidya, Ashwini",
    booktitle = "Proceedings of the Second workshop on Challenges in Processing {S}outh {A}sian Languages ({CH}i{PSAL}2026)",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/revision-workflow/2026.chipsal-1.31/",
    doi = "10.63317/2womrc9pc2b7",
    pages = "300--307",
    abstract = "Hate speech detection in Devanagari-scripted social media memes presents compounded challenges: multimodal content structure, script-specific linguistic complexity, and extreme data scarcity in low-resource settings. This paper presents our system for the CHiPSAL 2026 shared task, addressing both Subtask A (binary hate speech detection) and Subtask B (three-class sentiment classification: positive, neutral, negative). We propose a hybrid cross-modal attention fusion architecture that combines CLIP (ViT-B/32) for visual encoding with BGE-M3 for multilingual text representation, connected through 4-head self-attention and a learnable gating network that dynamically weights modality contributions on a per-sample basis. Systematic evaluation across eight model configurations demonstrates that explicit cross-modal reasoning achieves a 5.9{\%} F1-macro improvement over text-only baselines on Subtask A, while uncovering two unexpected but critical findings: English-centric vision models exhibit near-random performance on Devanagari script, and standard ensemble methods catastrophically degrade under data scarcity (N $\approx$ 850 per fold) due to correlated overfitting. Code and implementation details are available at a repository that has been anonymized for the review process and will be fully disclosed in the final version"
}