@inproceedings{singh-etal-2025-quality,
    title = "Quality Estimation and Post-Editing Using {LLM}s For {I}ndic Languages: How Good Is It?",
    author = "Singh, Anushka  and
      Pakhale, Aarya  and
      Khapra, Mitesh M.  and
      Dabre, Raj",
    editor = "Bouillon, Pierrette  and
      Gerlach, Johanna  and
      Girletti, Sabrina  and
      Volkart, Lise  and
      Rubino, Raphael  and
      Sennrich, Rico  and
      Farinha, Ana C.  and
      Gaido, Marco  and
      Daems, Joke  and
      Kenny, Dorothy  and
      Moniz, Helena  and
      Szoc, Sara",
    booktitle = "Proceedings of Machine Translation Summit XX: Volume 1",
    month = jun,
    year = "2025",
    address = "Geneva, Switzerland",
    publisher = "European Association for Machine Translation",
    url = "https://preview.aclanthology.org/mtsummit-25-ingestion/2025.mtsummit-1.30/",
    pages = "388--398",
    ISBN = "978-2-9701897-0-1",
    abstract = "Recently, there have been increasing efforts on Quality Estimation (QE) and Post-Editing (PE) using Large Language Models (LLMs) for Machine Translation (MT). However, the focus has mainly been on high resource languages and the approaches either rely on prompting or combining existing QE models with LLMs, instead of single end-to-end systems. In this paper, we investigate the efficacy of end-to-end QE and PE systems for low-resource languages taking 5 Indian languages as a use-case. We augment existing QE data containing multidimentional quality metric (MQM) error annotations with explanations of errors and PEs with the help of proprietary LLMs (GPT-4), following which we fine-tune Gemma-2-9B, an open-source multilingual LLM to perform QE and PE jointly. While our models attain QE capabilities competitive with or surpassing existing models in both referenceful and referenceless settings, we observe that they still struggle with PE. Further investigation reveals that this occurs because our models lack the ability to accurately identify fine-grained errors in the translation, despite being excellent indicators of overall quality. This opens up opportunities for research in end-to-end QE and PE for low-resource languages."
}