@inproceedings{coggins-etal-2025-aint,
    title = "That Ain{'}t Right: Assessing {LLM} Performance on {QA} in {A}frican {A}merican and {W}est {A}frican {E}nglish Dialects",
    author = "Coggins, William  and
      McKenzie, Jasmine  and
      Youm, Sangpil  and
      Mummaleti, Pradham  and
      Gilbert, Juan  and
      Ragan, Eric  and
      Dorr, Bonnie J",
    editor = "Zhang, Chen  and
      Allaway, Emily  and
      Shen, Hua  and
      Miculicich, Lesly  and
      Li, Yinqiao  and
      M'hamdi, Meryem  and
      Limkonchotiwat, Peerat  and
      Bai, Richard He  and
      T.y.s.s., Santosh  and
      Han, Sophia Simeng  and
      Thapa, Surendrabikram  and
      Rim, Wiem Ben",
    booktitle = "Proceedings of the 9th Widening NLP Workshop",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.winlp-main.21/",
    pages = "123--129",
    ISBN = "979-8-89176-351-7",
    abstract = "As Large Language Models (LLMs) gain mainstream public usage, understanding how users interact with them becomes increasingly important. Limited variety in training data raises concerns about LLM reliability across different language inputs. To explore this, we test several LLMs using functionally equivalent prompts expressed in different English sublanguages. We frame this analysis using Question-Answer (QA) pairs, which allow us to detect and evaluate appropriate and anomalous model behavior. We contribute a cross-LLM testing method and a new QA dataset translated into AAVE and WAPE variants. Early results reveal a notable drop in accuracy for one sublanguage relative to the baseline."
}