Abstract
In this work, we analyze the robustness of neural machine translation systems towards grammatical perturbations in the source. In particular, we focus on morphological inflection related perturbations. While this has been recently studied for English→French (MORPHEUS) (Tan et al., 2020), it is unclear how this extends to Any→English translation systems. We propose MORPHEUS-MULTILINGUAL that utilizes UniMorph dictionaries to identify morphological perturbations to source that adversely affect the translation models. Along with an analysis of state-of-the-art pretrained MT systems, we train and analyze systems for 11 language pairs using the multilingual TED corpus (Qi et al., 2018). We also compare this to actual errors of non-native speakers using Grammatical Error Correction datasets. Finally, we present a qualitative and quantitative analysis of the robustness of Any→English translation systems.- Anthology ID:
- 2021.sigmorphon-1.6
- Volume:
- Proceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
- Month:
- August
- Year:
- 2021
- Address:
- Online
- Editors:
- Garrett Nicolai, Kyle Gorman, Ryan Cotterell
- Venue:
- SIGMORPHON
- SIG:
- SIGMORPHON
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 49–59
- Language:
- URL:
- https://aclanthology.org/2021.sigmorphon-1.6
- DOI:
- 10.18653/v1/2021.sigmorphon-1.6
- Cite (ACL):
- Sai Muralidhar Jayanthi and Adithya Pratapa. 2021. A Study of Morphological Robustness of Neural Machine Translation. In Proceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 49–59, Online. Association for Computational Linguistics.
- Cite (Informal):
- A Study of Morphological Robustness of Neural Machine Translation (Jayanthi & Pratapa, SIGMORPHON 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2021.sigmorphon-1.6.pdf
- Code
- murali1996/morpheus_multilingual