@inproceedings{gao-etal-2025-tlue,
    title = "{TLUE}: A {T}ibetan Language Understanding Evaluation Benchmark",
    author = "Gao, Fan  and
      Huang, Cheng  and
      Liu, Yutong  and
      Tashi, Nyima  and
      Wang, Xiangxiang  and
      Tsering, Thupten  and
      Ma-bao, Ban  and
      Duojie, Renzeng  and
      Luosang, Gadeng  and
      Dongrub, Rinchen  and
      Tashi, Dorje  and
      Cd, Xiao Feng  and
      Yu, Yongbin  and
      Wang, Hao",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1777/",
    pages = "35059--35085",
    ISBN = "979-8-89176-332-6",
    abstract = "Large language models have made tremendous progress in recent years, but low-resource languages, like Tibetan, remain significantly underrepresented in their evaluation. Despite Tibetan being spoken by over seven million people, it has largely been neglected in the development and assessment of LLMs. To address this gap, we present a Tibetan Language Understanding Evaluation Benchmark, TLUE, which is also the first large-scale benchmark for measuring the proficiency of large language models in the Tibetan language. TLUE comprises two major components: a comprehensive multi-task understanding benchmark spanning 5 domains and 67 subdomains, and a safety benchmark encompassing 7 subdomains. Finally, we evaluate a diverse set of state-of-the-art LLMs. Experimental results demonstrate that most large language models perform below the random baseline, especially highlighting the considerable challenges they face in Tibetan language processing. TLUE provides a crucial foundation for advancing future research in Tibetan language understanding and highlights the importance of promoting greater inclusivity in the development of large language models."
}