Online Learning over Time in Adaptive Neural Machine Translation
Thierry Etchegoyhen, David Ponce, Harritxu Gete, Victor Ruiz
Abstract
Adaptive Machine Translation purports to dynamically include user feedback to improve translation quality. In a post-editing scenario, user corrections of machine translation output are thus continuously incorporated into translation models, reducing or eliminating repetitive error editing and increasing the usefulness of automated translation. In neural machine translation, this goal may be achieved via online learning approaches, where network parameters are updated based on each new sample. This type of adaptation typically requires higher learning rates, which can affect the quality of the models over time. Alternatively, less aggressive online learning setups may preserve model stability, at the cost of reduced adaptation to user-generated corrections. In this work, we evaluate different online learning configurations over time, measuring their impact on user-generated samples, as well as separate in-domain and out-of-domain datasets. Results in two different domains indicate that mixed approaches combining online learning with periodic batch fine-tuning might be needed to balance the benefits of online learning with model stability.- Anthology ID:
- 2021.ranlp-1.47
- Volume:
- Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
- Month:
- September
- Year:
- 2021
- Address:
- Held Online
- Editors:
- Ruslan Mitkov, Galia Angelova
- Venue:
- RANLP
- SIG:
- Publisher:
- INCOMA Ltd.
- Note:
- Pages:
- 411–420
- Language:
- URL:
- https://aclanthology.org/2021.ranlp-1.47
- DOI:
- Cite (ACL):
- Thierry Etchegoyhen, David Ponce, Harritxu Gete, and Victor Ruiz. 2021. Online Learning over Time in Adaptive Neural Machine Translation. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 411–420, Held Online. INCOMA Ltd..
- Cite (Informal):
- Online Learning over Time in Adaptive Neural Machine Translation (Etchegoyhen et al., RANLP 2021)
- PDF:
- https://preview.aclanthology.org/bionlp-24-ingestion/2021.ranlp-1.47.pdf