@inproceedings{zhou-etal-2020-improving,
    title = "Improving Autoregressive {NMT} with Non-Autoregressive Model",
    author = "Zhou, Long  and
      Zhang, Jiajun  and
      Zong, Chengqing",
    editor = "Wu, Hua  and
      Cherry, Colin  and
      Huang, Liang  and
      He, Zhongjun  and
      Liberman, Mark  and
      Cross, James  and
      Liu, Yang",
    booktitle = "Proceedings of the First Workshop on Automatic Simultaneous Translation",
    month = jul,
    year = "2020",
    address = "Seattle, Washington",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2020.autosimtrans-1.4/",
    doi = "10.18653/v1/2020.autosimtrans-1.4",
    pages = "24--29",
    abstract = "Autoregressive neural machine translation (NMT) models are often used to teach non-autoregressive models via knowledge distillation. However, there are few studies on improving the quality of autoregressive translation (AT) using non-autoregressive translation (NAT). In this work, we propose a novel Encoder-NAD-AD framework for NMT, aiming at boosting AT with global information produced by NAT model. Specifically, under the semantic guidance of source-side context captured by the encoder, the non-autoregressive decoder (NAD) first learns to generate target-side hidden state sequence in parallel. Then the autoregressive decoder (AD) performs translation from left to right, conditioned on source-side and target-side hidden states. Since AD has global information generated by low-latency NAD, it is more likely to produce a better translation with less time delay. Experiments on WMT14 En-De, WMT16 En-Ro, and IWSLT14 De-En translation tasks demonstrate that our framework achieves significant improvements with only 8{\%} speed degeneration over the autoregressive NMT."
}