Factors Affecting Translation Quality in In-context Learning for Multilingual Medical Domain

Jonathan Mutal, Raphael Rubino, Pierrette Bouillon


Abstract
Multilingual machine translation in the medical domain presents critical challenges due to limited parallel data, domain-specific terminology, and the high stakes associated with translation accuracy. In this paper, we explore the potential of in-context learning (ICL) with general-purpose large language models (LLMs) as an alternative to fine-tuning. Focusing on the medical domain and low-resource languages, we evaluate an instruction-tuned LLM on a translation task across 16 languages. We address four research questions centered on prompt design, examining the impact of the number of examples, the domain and register of examples, and the example selection strategy. Our results show that prompting with one to three examples from the same register and domain as the test input leads to the largest improvements in translation quality, as measured by automatic metrics, while translation quality gains plateau with an increased number of examples. Furthermore, we find that example selection methods - lexical and embedding based - do not yield significant benefits over random selection if the register of selected examples does not match that of the test input.
Anthology ID:
2025.wmt-1.9
Volume:
Proceedings of the Tenth Conference on Machine Translation
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Barry Haddow, Tom Kocmi, Philipp Koehn, Christof Monz
Venue:
WMT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
161–179
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.wmt-1.9/
DOI:
Bibkey:
Cite (ACL):
Jonathan Mutal, Raphael Rubino, and Pierrette Bouillon. 2025. Factors Affecting Translation Quality in In-context Learning for Multilingual Medical Domain. In Proceedings of the Tenth Conference on Machine Translation, pages 161–179, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Factors Affecting Translation Quality in In-context Learning for Multilingual Medical Domain (Mutal et al., WMT 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.wmt-1.9.pdf