ltzGLUE: Luxembourgish General Language Understanding Evaluation

Alistair Plum, Felicia K\"orner, Anne-Marie Lutgen, Laura Bernardy, Fred Philippy, Emilia Milano, Nils Rehlinger, Cedric Lothritz, Tharindu Ranasinghe, Barbara Plank, Christoph Purschke


Abstract
This paper presents ltzGLUE, the first Natural Language Understanding (NLU) benchmark for Luxembourgish (LTZ) based on the popular GLUE benchmark for English. Although NLU tasks are available for many european languages nowadays, LTZ is one of the official national languages that is often overlooked. We introduce new tasks and reuse existing ones to introduce the first official NLU benchmark and accompanying evaluation of encoder models for the language. Our tasks include common natural language processing tasks in binary and multi-class classification settings, including named entity recognition, topic classification, and intent classification. We evaluate various pre-trained language models for LTZ to present an overview of the current capabilities of these models on the LTZ language.
Anthology ID:
2026.findings-acl.476
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9791–9807
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.476/
DOI:
Bibkey:
Cite (ACL):
Alistair Plum, Felicia K\"orner, Anne-Marie Lutgen, Laura Bernardy, Fred Philippy, Emilia Milano, Nils Rehlinger, Cedric Lothritz, Tharindu Ranasinghe, Barbara Plank, and Christoph Purschke. 2026. ltzGLUE: Luxembourgish General Language Understanding Evaluation. In Findings of the Association for Computational Linguistics: ACL 2026, pages 9791–9807, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
ltzGLUE: Luxembourgish General Language Understanding Evaluation (Plum et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.476.pdf
Checklist:
 2026.findings-acl.476.checklist.pdf