@inproceedings{umar-etal-2026-anchoring,
    title = "Anchoring the Judge: Curriculum-Based Adaptation and Reference-Anchored {MQM} for {LLM}-Based Machine Translation of an Unseen Low-Resource Language - A Case of Nupe",
    author = "Umar, Umar Baba  and
      Bashir, Sulaimon Adebayo  and
      Mohammed, Abdulmalik Danlami",
    editor = "Hettiarachchi, Hansi  and
      Ranasinghe, Tharindu  and
      Plum, Alistair  and
      Rayson, Paul  and
      Mitkov, Ruslan  and
      Gaber, Mohamed  and
      Premasiri, Damith  and
      Tan, Fiona Anting  and
      Uyangodage, Lasitha",
    booktitle = "Proceedings of the Second Workshop on Language Models for Low-Resource Languages ({L}o{R}es{LM} 2026)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/manual-author-scripts/2026.loreslm-1.19/",
    pages = "200--211",
    ISBN = "979-8-89176-377-7",
    abstract = "Adapting large language models (LLMs) for machine translation has shown strong performance in low-resource languages; however, their effectiveness for \textit{unseen, extremely low-resource languages} remains largely unexplored. We present \textbf{NupeMT-QLoRA}, a curriculum-based adaptation framework for the Nupe{--}English language pair. Our approach employs a two-stage QLoRA fine-tuning strategy: (i) initial training on 34k noisy parallel sentence pairs, followed by (ii) continued fine-tuning on a smaller, cleaner set of 12k bidirectional parallel sentences with explicit translation-direction tags. This staged curriculum stabilizes optimization and improves robustness under severe data scarcity.We further identify a reliability crisis in existing automatic evaluation metrics for unseen languages. Popular LLM-based judges such as GEMBA and xCOMET exhibit weak correlation with human judgments (Kendall{'}s $\tau \approx 0.21$) and low inter-rater reliability (Fleiss' $\kappa \approx 0.27$), largely due to fluency bias. To address this, we propose \textbf{Ref-Anchor-MQM}, a reference-anchored evaluation protocol that forces the judge to extract Key Semantic Units from a human reference before scoring.Experimental results show that NupeMT-QLoRA substantially outperforms NLLB-200, improving chrF++ from 22.73 to 41.10, while Ref-Anchor-MQM achieves significantly higher alignment with human evaluation ($\tau = 0.71$). Our framework provides a scalable pipeline for adapting and evaluating LLMs on languages with zero prior representation."
}