@inproceedings{singh-etal-2025-lets,
    title = "Let{'}s Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports",
    author = "Singh, Punit Kumar  and
      Kumar, Nishant  and
      Ghosh, Akash  and
      Pasad, Kunal  and
      Soni, Khushi  and
      Jaishwal, Manisha  and
      Saha, Sriparna  and
      Alfarozi, Syukron Abu Ishaq  and
      Abagissa, Asres Temam  and
      Pasupa, Kitsuchart  and
      Yang, Haiqin  and
      Moreno, Jose G",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.769/",
    pages = "15205--15252",
    ISBN = "979-8-89176-332-6",
    abstract = "Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. To address this gap, we introduce \textbf{ \textit{CultSportQA}}, a benchmark designed to assess LMs' understanding of traditional sports across 60 countries and 6 continents, encompassing four distinct cultural categories. The dataset features 33,000 multiple-choice questions (MCQs) across text and image modalities, categorized into primarily three key types: history-based, rule-based, and scenario-based. To evaluate model performance, we employ zero-shot, few-shot, and chain-of-thought (CoT) prompting across a diverse set of Large Language Models (LLMs), Small Language Models (SLMs), and Multimodal Large Language Models (MLMs). By providing a comprehensive multilingual and multicultural sports benchmark, \textbf{ \textit{CultSportQA}} establishes a new standard for assessing AI{'}s ability to understand and reason about traditional sports. The dataset will be publicly available, fostering research in culturally aware AI systems."
}