@inproceedings{chennuru-adebayo-2026-neutral,
    title = "When Neutral Turns Negative: Cross-Domain Failure Modes in {H}inglish Political Sentiment Analysis",
    author = "Chennuru, Rahul  and
      Adebayo, Kolawole John",
    editor = "Afli, Haithem  and
      Bouamor, Houda  and
      Zaghouani, Wajdi  and
      Ghannay, Sahar  and
      Hossain, Shehenaz",
    booktitle = "Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences ({P}olitical{NLP} 2026)",
    month = may,
    year = "2026",
    address = "Palma, Mallorca (Spain)",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/test-year-match/2026.politicalnlp-1.24/",
    doi = "10.63317/3evceg5aip7m",
    pages = "219--227",
    abstract = "Sentiment analysis models are increasingly deployed to analyze political discourse, yet strong in-domain performance does not guarantee robustness under domain shift. We study cross-domain generalization in Hinglish (Hindi{--}English code-mixed) sentiment analysis by evaluating a fine-tuned XLM-RoBERTa classifier, trained on 29,000 general-domain Hinglish sentences, on a curated benchmark of politically oriented Hinglish text. While the model achieves 92.02{\%} accuracy in-domain, performance drops to 71.83{\%} under political domain shift. Error analysis reveals a pronounced directional bias with 48.9{\%} of neutral political statements misclassified as negative, indicating a systematic neutrality-to-negative shift. In addition, 87.5{\%} of incorrect predictions are assigned confidence scores above 95{\%}, pointing to severe miscalibration under distribution shift. We further compare these results against an instruction-tuned large language model (Llama 3.3), which achieves 90.85{\%} zero-shot accuracy and 94.37{\%} accuracy with contextual prompting, while substantially reducing neutrality bias. Our findings indicate the need for domain-aware evaluation, calibration diagnostics, and explicit reporting of failure modes when deploying sentiment models in politically sensitive settings."
}