Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019
Abstract
This paper describes the NICT-2 neural machine translation system at the 6th Workshop on Asian Translation. This system employs the standard Transformer model but features the following two characteristics. One is the long warm-up strategy, which performs a longer warm-up of the learning rate at the start of the training than conventional approaches. Another is that the system introduces self-training approaches based on multiple back-translations generated by sampling. We participated in three tasks—ASPEC.en-ja, ASPEC.ja-en, and TDDC.ja-en—using this system.- Anthology ID:
- D19-5217
- Volume:
- Proceedings of the 6th Workshop on Asian Translation
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Toshiaki Nakazawa, Chenchen Ding, Raj Dabre, Anoop Kunchukuttan, Nobushige Doi, Yusuke Oda, Ondřej Bojar, Shantipriya Parida, Isao Goto, Hidaya Mino
- Venue:
- WAT
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 141–146
- Language:
- URL:
- https://aclanthology.org/D19-5217
- DOI:
- 10.18653/v1/D19-5217
- Cite (ACL):
- Kenji Imamura and Eiichiro Sumita. 2019. Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019. In Proceedings of the 6th Workshop on Asian Translation, pages 141–146, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019 (Imamura & Sumita, WAT 2019)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/D19-5217.pdf