@inproceedings{yan-etal-2025-nvidia,
    title = "Nvidia-Nemo{'}s {WMT} 2025 Metrics Shared Task Submission",
    author = "Yan, Brian  and
      Ding, Shuoyang  and
      Wang, Kuang-Da  and
      Ouyang, Siqi  and
      Hrinchuk, Oleksii  and
      Lavrukhin, Vitaly  and
      Ginsburg, Boris",
    editor = "Haddow, Barry  and
      Kocmi, Tom  and
      Koehn, Philipp  and
      Monz, Christof",
    booktitle = "Proceedings of the Tenth Conference on Machine Translation",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.wmt-1.66/",
    pages = "920--925",
    ISBN = "979-8-89176-341-8",
    abstract = "This paper describes Nvidia-Nemo{'}s WMT 2025 Metrics Shared Task submission. We investigated two strategies for extending Machine Translation (MT) evaluation to unsegmented documents: 1) first segmenting into sentences and then applying regression-based metrics and 2) directly utilizing the long-context capabilities of LLMs. The base comparison of the segmentation-based and LLM-based metrics on the WMT 2023-24 evaluation sets indicated that the former performs more robustly across language pairs.Thus we sought to improve the LLM-based approach by incorporating relative evaluation - this setting jointly evaluates all candidate translations at once and relative to each other, rather than evaluating each separately. Our experiments using the open-source Qwen3 LLM show that relative evaluation improves score correlations with human judgment, but only if the task is structured as a 2-stage evaluate-then-refine problem."
}