@inproceedings{thapa-etal-2025-probing,
    title = "Probing the Limits of Multilingual Language Understanding: Low-Resource Language Proverbs as {LLM} Benchmark for {AI} Wisdom",
    author = "Thapa, Surendrabikram  and
      Rauniyar, Kritesh  and
      Veeramani, Hariram  and
      Adhikari, Surabhi  and
      Razzak, Imran  and
      Naseem, Usman",
    editor = "Strube, Michael  and
      Braud, Chloe  and
      Hardmeier, Christian  and
      Li, Junyi Jessy  and
      Loaiciga, Sharid  and
      Zeldes, Amir  and
      Li, Chuyuan",
    booktitle = "Proceedings of the 6th Workshop on Computational Approaches to Discourse, Context and Document-Level Inferences (CODI 2025)",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/name-variant-enfa-fane/2025.codi-1.11/",
    doi = "10.18653/v1/2025.codi-1.11",
    pages = "120--129",
    ISBN = "979-8-89176-343-2",
    abstract = "Understanding and interpreting culturally specific language remains a significant challenge for multilingual natural language processing (NLP) systems, particularly for less-resourced languages. To address this problem, this paper introduces PRONE, a novel dataset of 2,830 Nepali proverbs, and evaluates the performance of various language models (LMs) in two tasks: (i) identifying the correct meaning of a proverb from multiple choices, and (ii) categorizing proverbs into predefined thematic categories. The models, including both open-source and proprietary, were tested in zero-shot and few-shot settings with prompts in English and Nepali. While models like GPT-4o demonstrated promising results and achieved the highest performance among LMs, they still fall short of human-level accuracy in understanding and categorizing culturally nuanced content, highlighting the need for more inclusive NLP."
}