Abstract
Latent alignment objectives such as CTC and AXE significantly improve non-autoregressive machine translation models. Can they improve autoregressive models as well? We explore the possibility of training autoregressive machine translation models with latent alignment objectives, and observe that, in practice, this approach results in degenerate models. We provide a theoretical explanation for these empirical results, and prove that latent alignment objectives are incompatible with teacher forcing.- Anthology ID:
- 2021.naacl-main.209
- Volume:
- Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Month:
- June
- Year:
- 2021
- Address:
- Online
- Editors:
- Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2637–2641
- Language:
- URL:
- https://aclanthology.org/2021.naacl-main.209
- DOI:
- 10.18653/v1/2021.naacl-main.209
- Cite (ACL):
- Adi Haviv, Lior Vassertail, and Omer Levy. 2021. Can Latent Alignments Improve Autoregressive Machine Translation?. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2637–2641, Online. Association for Computational Linguistics.
- Cite (Informal):
- Can Latent Alignments Improve Autoregressive Machine Translation? (Haviv et al., NAACL 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2021.naacl-main.209.pdf