Abstract
We present a deep generative model of bilingual sentence pairs for machine translation. The model generates source and target sentences jointly from a shared latent representation and is parameterised by neural networks. We perform efficient training using amortised variational inference and reparameterised gradients. Additionally, we discuss the statistical implications of joint modelling and propose an efficient approximation to maximum a posteriori decoding for fast test-time predictions. We demonstrate the effectiveness of our model in three machine translation scenarios: in-domain training, mixed-domain training, and learning from a mix of gold-standard and synthetic data. Our experiments show consistently that our joint formulation outperforms conditional modelling (i.e. standard neural machine translation) in all such scenarios.- Anthology ID:
- W19-4315
- Volume:
- Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Isabelle Augenstein, Spandana Gella, Sebastian Ruder, Katharina Kann, Burcu Can, Johannes Welbl, Alexis Conneau, Xiang Ren, Marek Rei
- Venue:
- RepL4NLP
- SIG:
- SIGREP
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 124–141
- Language:
- URL:
- https://aclanthology.org/W19-4315
- DOI:
- 10.18653/v1/W19-4315
- Cite (ACL):
- Bryan Eikema and Wilker Aziz. 2019. Auto-Encoding Variational Neural Machine Translation. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 124–141, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Auto-Encoding Variational Neural Machine Translation (Eikema & Aziz, RepL4NLP 2019)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/W19-4315.pdf
- Code
- Roxot/AEVNMT
- Data
- WMT 2016