@inproceedings{guo-nguyen-2020-document,
    title = "Document-Level Neural Machine Translation Using {BERT} as Context Encoder",
    author = "Guo, Zhiyu  and
      Nguyen, Minh Le",
    editor = "Shmueli, Boaz  and
      Huang, Yin Jou",
    booktitle = "Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: Student Research Workshop",
    month = dec,
    year = "2020",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2020.aacl-srw.15/",
    doi = "10.18653/v1/2020.aacl-srw.15",
    pages = "101--107",
    abstract = "Large-scale pre-trained representations such as BERT have been widely used in many natural language understanding tasks. The methods of incorporating BERT into document-level machine translation are still being explored. BERT is able to understand sentence relationships since BERT is pre-trained using the next sentence prediction task. In our work, we leverage this property to improve document-level machine translation. In our proposed model, BERT performs as a context encoder to achieve document-level contextual information, which is then integrated into both the encoder and decoder. Experiment results show that our proposed method can significantly outperform strong document-level machine translation baselines on BLEU score. Moreover, the ablation study shows our method can capture document-level context information to boost translation performance."
}