@inproceedings{khanal-2026-zeror,
    title = "{Z}ero{R}@{CH}i{PSAL} 2026: Two-Stage Vision-Language Adaptation with Contrastive Learning for {N}epali Meme Classification",
    author = "Khanal, Nitiz",
    editor = "Sarveswaran, Kengatharaiyer  and
      Vaidya, Ashwini",
    booktitle = "Proceedings of the Second workshop on Challenges in Processing {S}outh {A}sian Languages ({CH}i{PSAL}2026)",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/test-year-match/2026.chipsal-1.28/",
    doi = "10.63317/5nypaugdk6kz",
    pages = "275--283",
    abstract = "This paper presents our system for the CHiPSAL 2026 shared task on multimodal hate speech and sentiment detection in Nepali memes. We address both subtasks: binary hate speech classification and three-class sentiment analysis. Our approach adapts the Robust Adaptation of Hateful Meme Detection (RA-HMD) framework using Qwen3-VL-8B-Instruct, a state-of-the-art vision-language model with native Devanagari support. We employ a two-stage training pipeline: (1) LoRA fine-tuning with an MLP projection head for generative classification, and (2) contrastive backbone fine-tuning with supervised InfoNCE loss. We handle class imbalance through minority oversampling, image augmentation, and focal loss. At inference, we ensemble Stage 1 token probabilities with Stage 2 classifier scores using validation-tuned weights. Our end-to-end approach eliminates error propagation from separate OCR and translation pipelines by leveraging the model{'}s native Devanagari understanding. Our system achieved 2nd place on hate speech detection (F1: 0.797) and 4th place on sentiment analysis (F1: 0.518). We provide detailed ablations, error analysis, and insights into adapting large vision-language models for low-resource South Asian languages."
}