Multilingual NMT with a Language-Independent Attention Bridge
Raúl Vázquez, Alessandro Raganato, Jörg Tiedemann, Mathias Creutz
Abstract
In this paper, we propose an architecture for machine translation (MT) capable of obtaining multilingual sentence representations by incorporating an intermediate attention bridge that is shared across all languages. We train the model with language-specific encoders and decoders that are connected through an inner-attention layer on the encoder side. The attention bridge exploits the semantics from each language for translation and develops into a language-agnostic meaning representation that can efficiently be used for transfer learning. We present a new framework for the efficient development of multilingual neural machine translation (NMT) using this model and scheduled training. We have tested the approach in a systematic way with a multi-parallel data set. The model achieves substantial improvements over strong bilingual models and performs well for zero-shot translation, which demonstrates its ability of abstraction and transfer learning.- Anthology ID:
- W19-4305
- Volume:
- Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Isabelle Augenstein, Spandana Gella, Sebastian Ruder, Katharina Kann, Burcu Can, Johannes Welbl, Alexis Conneau, Xiang Ren, Marek Rei
- Venue:
- RepL4NLP
- SIG:
- SIGREP
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 33–39
- Language:
- URL:
- https://preview.aclanthology.org/icon-24-ingestion/W19-4305/
- DOI:
- 10.18653/v1/W19-4305
- Cite (ACL):
- Raúl Vázquez, Alessandro Raganato, Jörg Tiedemann, and Mathias Creutz. 2019. Multilingual NMT with a Language-Independent Attention Bridge. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 33–39, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Multilingual NMT with a Language-Independent Attention Bridge (Vázquez et al., RepL4NLP 2019)
- PDF:
- https://preview.aclanthology.org/icon-24-ingestion/W19-4305.pdf
- Data
- Multi30K