Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation
Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, Marjan Ghazvininejad
Abstract
Contemporary machine translation systems achieve greater coverage by applying subword models such as BPE and character-level CNNs, but these methods are highly sensitive to orthographical variations such as spelling mistakes. We show how training on a mild amount of random synthetic noise can dramatically improve robustness to these variations, without diminishing performance on clean text. We focus on translation performance on natural typos, and show that robustness to such noise can be achieved using a balanced diet of simple synthetic noises at training time, without access to the natural noise data or distribution.- Anthology ID:
- D19-5506
- Volume:
- Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019)
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Venue:
- WNUT
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 42–47
- Language:
- URL:
- https://aclanthology.org/D19-5506
- DOI:
- 10.18653/v1/D19-5506
- Cite (ACL):
- Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, and Marjan Ghazvininejad. 2019. Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation. In Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019), pages 42–47, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation (Karpukhin et al., WNUT 2019)
- PDF:
- https://preview.aclanthology.org/starsem-semeval-split/D19-5506.pdf
- Data
- MTNT