@inproceedings{yu-etal-2025-injongo,
    title = "{INJONGO}: A Multicultural Intent Detection and Slot-filling Dataset for 16 {A}frican Languages",
    author = "Yu, Hao  and
      Alabi, Jesujoba Oluwadara  and
      Bukula, Andiswa  and
      Zhuang, Jian Yun  and
      Lee, En-Shiun Annie  and
      Guge, Tadesse Kebede  and
      Azime, Israel Abebe  and
      Buzaaba, Happy  and
      Sibanda, Blessing Kudzaishe  and
      Kalipe, Godson Koffi  and
      Mukiibi, Jonathan  and
      Kabongo Kabenamualu, Salomon  and
      Setaka, Mmasibidi  and
      Ndolela, Lolwethu  and
      Odu, Nkiruka  and
      Mabuya, Rooweither  and
      Muhammad, Shamsuddeen Hassan  and
      Osei, Salomey  and
      Samb, Sokhar  and
      Klakow, Dietrich  and
      Adelani, David Ifeoluwa",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.464/",
    pages = "9429--9452",
    ISBN = "979-8-89176-251-0",
    abstract = "Slot-filling and intent detection are well-established tasks in Conversational AI. However, current large-scale benchmarks for these tasks often exclude evaluations of low-resource languages and rely on translations from English benchmarks, thereby predominantly reflecting Western-centric concepts. In this paper, we introduce ``INJONGO'' - a multicultural, open-source benchmark dataset for 16 African languages with utterances generated by native speakers across diverse domains, including banking, travel, home, and dining. Through extensive experiments, we benchmark fine-tuning multilingual transformer models and prompting large language models (LLMs), and show the advantage of leveraging African-cultural utterances over Western-centric utterances for improving cross-lingual transfer from the English language. Experimental results reveal that current LLMs struggle with the slot-filling task, with GPT-4o achieving an average performance of 26 F1. In contrast, intent detection performance is notably better, with an average accuracy of 70.6{\%}, though it still falls short of fine-tuning baselines. When compared to the English language, GPT-4o and fine-tuning baselines perform similarly on intent detection, achieving an accuracy of approximately 81{\%}. Our findings suggest that LLMs performance is still behind for many low-resource African languages, and more work is needed to further improve their downstream performance."
}