@inproceedings{vamvas-etal-2025-expanding,
    title = "Expanding the {WMT}24++ Benchmark with Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, and Vallader",
    author = "Vamvas, Jannis  and
      P{\'e}rez Prat, Ignacio  and
      Soliva, Not  and
      Baltermia-Guetg, Sandra  and
      Beeli, Andrina  and
      Beeli, Simona  and
      Capeder, Madlaina  and
      Decurtins, Laura  and
      Gregori, Gian Peder  and
      Hobi, Flavia  and
      Holderegger, Gabriela  and
      Lazzarini, Arina  and
      Lazzarini, Viviana  and
      Rosselli, Walter  and
      Vital, Bettina  and
      Rutkiewicz, Anna  and
      Sennrich, Rico",
    editor = "Haddow, Barry  and
      Kocmi, Tom  and
      Koehn, Philipp  and
      Monz, Christof",
    booktitle = "Proceedings of the Tenth Conference on Machine Translation",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.wmt-1.79/",
    pages = "1028--1047",
    ISBN = "979-8-89176-341-8",
    abstract = "The Romansh language, spoken in Switzerland, has limited resources for machine translation evaluation. In this paper, we present a benchmark for six varieties of Romansh: Rumantsch Grischun, a supra-regional variety, and five regional varieties: Sursilvan, Sutsilvan, Surmiran, Puter, and Vallader. Our reference translations were created by human translators based on the WMT24++ benchmark, which ensures parallelism with more than 55 other languages. An automatic evaluation of existing MT systems and LLMs shows that translation out of Romansh into German is handled relatively well for all the varieties, but translation into Romansh is still challenging."
}