Robust Neural Machine Translation with Doubly Adversarial Inputs

Yong Cheng, Lu Jiang, Wolfgang Macherey


Abstract
Neural machine translation (NMT) often suffers from the vulnerability to noisy perturbations in the input. We propose an approach to improving the robustness of NMT models, which consists of two parts: (1) attack the translation model with adversarial source examples; (2) defend the translation model with adversarial target inputs to improve its robustness against the adversarial source inputs. For the generation of adversarial inputs, we propose a gradient-based method to craft adversarial examples informed by the translation loss over the clean inputs. Experimental results on Chinese-English and English-German translation tasks demonstrate that our approach achieves significant improvements (2.8 and 1.6 BLEU points) over Transformer on standard clean benchmarks as well as exhibiting higher robustness on noisy data.
Anthology ID:
P19-1425
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4324–4333
Language:
URL:
https://aclanthology.org/P19-1425
DOI:
10.18653/v1/P19-1425
Bibkey:
Cite (ACL):
Yong Cheng, Lu Jiang, and Wolfgang Macherey. 2019. Robust Neural Machine Translation with Doubly Adversarial Inputs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4324–4333, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Robust Neural Machine Translation with Doubly Adversarial Inputs (Cheng et al., ACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/P19-1425.pdf