@inproceedings{awobade-etal-2026-afrivox,
    title = "{A}fri{V}ox: Probing Multilingual and Accent Robustness of Speech {LLM}s",
    author = "Awobade, Busayo  and
      Sanni, Mardhiyah  and
      Abdullahi, Tassallah  and
      Okocha, Chibuzor  and
      Ezema, Kelechi  and
      Kayande, Devendra Deepak  and
      Ismaila, Lukman Enegi  and
      Olatunji, Tobi  and
      Katuka, Gloria Ashiya",
    editor = "Demberg, Vera  and
      Inui, Kentaro  and
      Marquez, Llu{\'i}s",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.122/",
    pages = "2672--2690",
    ISBN = "979-8-89176-380-7",
    abstract = "Recent advances in multimodal and speech-native large language models (LLMs) have delivered impressive speech recognition, translation, understanding, and question-answering capabilities for high-resource languages. However, African languages and non-native French or English accents remain dramatically underrepresented in benchmarks limiting the understanding and applicability of leading LLMs for millions of francophone and anglophone users in low-resource settings. We presents AfriVox, an open-source benchmark (including novel domain-specific and unscripted datasets) across 20 African languages, African-accented French, Arabic, and 100+ African English accents, contrasting leading multimodal speech LLMs with traditional unimodal automatic speech transcription (ASR) and translation (AST) models. Our analysis reveals significant language coverage variation, surprising LLM translation performance gains (e.g. Gemini), robustness concerns with unscripted speech, and substantial performance disparities for ``supported'' African languages. We profile the strengths, limitations, and language support of each model, and conduct the first targeted fine-tuning of a modern speech LLM (Qwen2.5-Omni) for three Nigerian languages, exceeding SOTA, and achieving up to 54{\%} relative WER reduction and significant BLEU gains, offering practical guidance for implementers seeking to serve local language users."
}