@inproceedings{zhou-keung-2020-improving,
    title = "Improving Non-autoregressive Neural Machine Translation with Monolingual Data",
    author = "Zhou, Jiawei  and
      Keung, Phillip",
    editor = "Jurafsky, Dan  and
      Chai, Joyce  and
      Schluter, Natalie  and
      Tetreault, Joel",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2020.acl-main.171/",
    doi = "10.18653/v1/2020.acl-main.171",
    pages = "1893--1898",
    abstract = "Non-autoregressive (NAR) neural machine translation is usually done via knowledge distillation from an autoregressive (AR) model. Under this framework, we leverage large monolingual corpora to improve the NAR model{'}s performance, with the goal of transferring the AR model{'}s generalization ability while preventing overfitting. On top of a strong NAR baseline, our experimental results on the WMT14 En-De and WMT16 En-Ro news translation tasks confirm that monolingual data augmentation consistently improves the performance of the NAR model to approach the teacher AR model{'}s performance, yields comparable or better results than the best non-iterative NAR methods in the literature and helps reduce overfitting in the training process."
}