Cheonbok Park
2022
DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation
Cheonbok Park
|
Hantae Kim
|
Ioan Calapodescu
|
Hyun Chang Cho
|
Vassilina Nikoulina
Findings of the Association for Computational Linguistics: ACL 2022
Domain Adaptation (DA) of Neural Machine Translation (NMT) model often relies on a pre-trained general NMT model which is adapted to the new domain on a sample of in-domain parallel data. Without parallel data, there is no way to estimate the potential benefit of DA, nor the amount of parallel samples it would require. It is however a desirable functionality that could help MT practitioners to make an informed decision before investing resources in dataset creation. We propose a Domain adaptation Learning Curve prediction (DaLC) model that predicts prospective DA performance based on in-domain monolingual samples in the source language. Our model relies on the NMT encoder representations combined with various instance and corpus-level features. We demonstrate that instance-level is better able to distinguish between different domains compared to corpus-level frameworks proposed in previous studies Finally, we perform in-depth analyses of the results highlighting the limitations of our approach, and provide directions for future research.
2021
Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning
Cheonbok Park
|
Yunwon Tae
|
TaeHee Kim
|
Soyoung Yang
|
Mohammad Azam Khan
|
Lucy Park
|
Jaegul Choo
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Unsupervised machine translation, which utilizes unpaired monolingual corpora as training data, has achieved comparable performance against supervised machine translation. However, it still suffers from data-scarce domains. To address this issue, this paper presents a novel meta-learning algorithm for unsupervised neural machine translation (UNMT) that trains the model to adapt to another domain by utilizing only a small amount of training data. We assume that domain-general knowledge is a significant factor in handling data-scarce domains. Hence, we extend the meta-learning algorithm, which utilizes knowledge learned from high-resource domains, to boost the performance of low-resource UNMT. Our model surpasses a transfer learning-based approach by up to 2-3 BLEU scores. Extensive experimental results show that our proposed algorithm is pertinent for fast adaptation and consistently outperforms other baselines.
Search
Co-authors
- Hantae Kim 1
- Ioan Calapodescu 1
- Hyun Chang Cho 1
- Vassilina Nikoulina 1
- Yunwon Tae 1
- show all...