@inproceedings{ochieng-etal-2026-reasoning,
    title = "Reasoning Beyond Labels: Measuring {LLM} Sentiment in Low-Resource, Culturally Nuanced Contexts",
    author = "Ochieng, Millicent  and
      Thieme, Anja  and
      Ezeani, Ignatius  and
      Ueno, Risa  and
      Maina, Samuel Chege  and
      Ronen, Keshet  and
      Gonzalez, Javier  and
      O'Neill, Jacki",
    editor = "Chimoto, Everlyn Asiko  and
      Lignos, Constantine  and
      Muhammad, Shamsuddeen  and
      Abdulmumin, Idris  and
      Siro, Clemencia  and
      Adelani, David Ifeoluwa",
    booktitle = "Proceedings of the 7th Workshop on {A}frican Natural Language Processing ({A}frica{NLP} 2026)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/manual-author-scripts/2026.africanlp-main.7/",
    pages = "64--81",
    ISBN = "979-8-89176-364-7",
    abstract = "Sentiment analysis in low-resource, culturally nuanced contexts challenges conventional NLP approaches that assume fixed labels and universal affective expressions. We present a diagnostic framework that treats sentiment as a context-dependent, culturally embedded construct, and evaluate how large language models (LLMs) reason about sentiment in informal, code-mixed WhatsApp messages from Nairobi youth health groups. Using human-annotated data, sentiment-flipped counterfactuals, and rubric-based explanation evaluation, we probe LLM interpretability, robustness, and alignment with human reasoning. Framing our evaluation through a social science measurement lens, we operationalize LLM outputs as an instrument for measuring the abstract concept of sentiment. Our findings reveal significant variation in model reasoning quality, with top-tier LLMs demonstrating greater interpretive stability, while smaller open-weight models in our study show reduced stability under ambiguity or sentiment shifts. This work highlights the need for culturally sensitive, reasoning-aware AI evaluation in complex, real-world communication."
}