Evolution Strategy Based Automatic Tuning of Neural Machine Translation Systems

Hao Qin, Takahiro Shinozaki, Kevin Duh


Abstract
Neural machine translation (NMT) systems have demonstrated promising results in recent years. However, non-trivial amounts of manual effort are required for tuning network architectures, training configurations, and pre-processing settings such as byte pair encoding (BPE). In this study, we propose an evolution strategy based automatic tuning method for NMT. In particular, we apply the covariance matrix adaptation-evolution strategy (CMA-ES), and investigate a Pareto-based multi-objective CMA-ES to optimize the translation performance and computational time jointly. Experimental results show that the proposed method automatically finds NMT systems that outperform the initial manual setting.
Anthology ID:
2017.iwslt-1.17
Volume:
Proceedings of the 14th International Conference on Spoken Language Translation
Month:
December 14-15
Year:
2017
Address:
Tokyo, Japan
Venue:
IWSLT
SIG:
SIGSLT
Publisher:
International Workshop on Spoken Language Translation
Note:
Pages:
120–128
Language:
URL:
https://aclanthology.org/2017.iwslt-1.17
DOI:
Bibkey:
Cite (ACL):
Hao Qin, Takahiro Shinozaki, and Kevin Duh. 2017. Evolution Strategy Based Automatic Tuning of Neural Machine Translation Systems. In Proceedings of the 14th International Conference on Spoken Language Translation, pages 120–128, Tokyo, Japan. International Workshop on Spoken Language Translation.
Cite (Informal):
Evolution Strategy Based Automatic Tuning of Neural Machine Translation Systems (Qin et al., IWSLT 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2017.iwslt-1.17.pdf