Abstract
Neural Machine Translation (NMT) is an open vocabulary problem. As a result, dealing with the words not occurring during training (a.k.a. out-of-vocabulary (OOV) words) have long been a fundamental challenge for NMT systems. The predominant method to tackle this problem is Byte Pair Encoding (BPE) which splits words, including OOV words, into sub-word segments. BPE has achieved impressive results for a wide range of translation tasks in terms of automatic evaluation metrics. While it is often assumed that by using BPE, NMT systems are capable of handling OOV words, the effectiveness of BPE in translating OOV words has not been explicitly measured. In this paper, we study to what extent BPE is successful in translating OOV words at the word-level. We analyze the translation quality of OOV words based on word type, number of segments, cross-attention weights, and the frequency of segment n-grams in the training data. Our experiments show that while careful BPE settings seem to be fairly useful in translating OOV words across datasets, a considerable percentage of OOV words are translated incorrectly. Furthermore, we highlight the slightly higher effectiveness of BPE in translating OOV words for special cases, such as named-entities and when the languages involved are linguistically close to each other.- Anthology ID:
- 2022.amta-research.9
- Volume:
- Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)
- Month:
- September
- Year:
- 2022
- Address:
- Orlando, USA
- Editors:
- Kevin Duh, Francisco Guzmán
- Venue:
- AMTA
- SIG:
- Publisher:
- Association for Machine Translation in the Americas
- Note:
- Pages:
- 117–130
- Language:
- URL:
- https://aclanthology.org/2022.amta-research.9
- DOI:
- Cite (ACL):
- Ali Araabi, Christof Monz, and Vlad Niculae. 2022. How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation?. In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 117–130, Orlando, USA. Association for Machine Translation in the Americas.
- Cite (Informal):
- How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation? (Araabi et al., AMTA 2022)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2022.amta-research.9.pdf
- Data
- FLoRes-101