Pavel Zubitski
2025
KazBench-KK: A Cultural-Knowledge Benchmark for Kazakh
Sanzhar Umbet
|
Sanzhar Murzakhmetov
|
Beksultan Sagyndyk
|
Kirill Yakunin
|
Timur Akishev
|
Pavel Zubitski
Proceedings of the Fourth Workshop on NLP Applications to Field Linguistics
We introduce KazBench-KK, a comprehensive 7,111-question multiple-choice benchmark designed to assess large language models’ understanding of culturally grounded Kazakh knowledge. By combining expert-curated topics with LLM-assisted web mining, we create a diverse dataset spanning 17 culturally salient domains, including pastoral traditions, social hierarchies, and contemporary politics. Beyond evaluation, KazBench-KK serves as a practical tool for field linguists, enabling rapid lexical elicitation, glossing, and topic prioritization. Our benchmarking of various open-source LLMs reveals that reinforcement-tuned models outperform others, but smaller, domain-focused fine-tunes can rival larger models in specific cultural contexts.