Quality-aware Neural Machine Translation with Self-evaluation

Jiajia Cui, Lingling Mu, Qiuhui Liu, Hongfei Xu


Abstract
"The performance of neural machine translation relies on a large amount of data, but crawled sentence pairs are of different quality. The low-quality sentence pairs may provide helpful translation knowledge but also teach the model to generate low-quality translations. Making the model aware of the quality of training instances may help the model distinguish between good and bad translations while leveraging the translation knowledge. In this paper, we evaluate the quality of training instances with the average per-token loss (negative log-likelihood) from translation mod-els, convert the quality scores into embeddings through vector interpolation and feed the quality embedding into the translation model during its training. We ask the model to decode with the best quality score to generate good translations during inference. Experiments on the IWSLT 14 German to English, WMT 14 English to German and WMT 22 English to Japanese translation tasks show that our method can effectively lead to consistent and significant improvements across multiple metrics."
Anthology ID:
2025.ccl-1.87
Volume:
Proceedings of the 24th China National Conference on Computational Linguistics (CCL 2025)
Month:
August
Year:
2025
Address:
Jinan, China
Editors:
Maosong Sun, Peiyong Duan, Zhiyuan Liu, Ruifeng Xu, Weiwei Sun
Venue:
CCL
SIG:
Publisher:
Chinese Information Processing Society of China
Note:
Pages:
1178–1187
Language:
URL:
https://preview.aclanthology.org/ingest-ccl/2025.ccl-1.87/
DOI:
Bibkey:
Cite (ACL):
Jiajia Cui, Lingling Mu, Qiuhui Liu, and Hongfei Xu. 2025. Quality-aware Neural Machine Translation with Self-evaluation. In Proceedings of the 24th China National Conference on Computational Linguistics (CCL 2025), pages 1178–1187, Jinan, China. Chinese Information Processing Society of China.
Cite (Informal):
Quality-aware Neural Machine Translation with Self-evaluation (Cui et al., CCL 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-ccl/2025.ccl-1.87.pdf