Dealing with the Paradox of Quality Estimation
Sugyeong Eo, Chanjun Park, Hyeonseok Moon, Jaehyung Seo, Heuiseok Lim
Abstract
In quality estimation (QE), the quality of translation can be predicted by referencing the source sentence and the machine translation (MT) output without access to the reference sentence. However, there exists a paradox in that constructing a dataset for creating a QE model requires non-trivial human labor and time, and it may even requires additional effort compared to the cost of constructing a parallel corpus. In this study, to address this paradox and utilize the various applications of QE, even in low-resource languages (LRLs), we propose a method for automatically constructing a pseudo-QE dataset without using human labor. We perform a comparative analysis on the pseudo-QE dataset using multilingual pre-trained language models. As we generate the pseudo dataset, we conduct experiments using various external machine translators as test sets to verify the accuracy of the results objectively. Also, the experimental results show that multilingual BART demonstrates the best performance, and we confirm the applicability of QE in LRLs using pseudo-QE dataset construction methods.- Anthology ID:
- 2021.mtsummit-loresmt.1
- Volume:
- Proceedings of the 4th Workshop on Technologies for MT of Low Resource Languages (LoResMT2021)
- Month:
- August
- Year:
- 2021
- Address:
- Virtual
- Editors:
- John Ortega, Atul Kr. Ojha, Katharina Kann, Chao-Hong Liu
- Venue:
- LoResMT
- SIG:
- Publisher:
- Association for Machine Translation in the Americas
- Note:
- Pages:
- 1–10
- Language:
- URL:
- https://aclanthology.org/2021.mtsummit-loresmt.1
- DOI:
- Cite (ACL):
- Sugyeong Eo, Chanjun Park, Hyeonseok Moon, Jaehyung Seo, and Heuiseok Lim. 2021. Dealing with the Paradox of Quality Estimation. In Proceedings of the 4th Workshop on Technologies for MT of Low Resource Languages (LoResMT2021), pages 1–10, Virtual. Association for Machine Translation in the Americas.
- Cite (Informal):
- Dealing with the Paradox of Quality Estimation (Eo et al., LoResMT 2021)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2021.mtsummit-loresmt.1.pdf