@inproceedings{khayrallah-etal-2018-regularized,
    title = "Regularized Training Objective for Continued Training for Domain Adaptation in Neural Machine Translation",
    author = "Khayrallah, Huda  and
      Thompson, Brian  and
      Duh, Kevin  and
      Koehn, Philipp",
    editor = "Birch, Alexandra  and
      Finch, Andrew  and
      Luong, Thang  and
      Neubig, Graham  and
      Oda, Yusuke",
    booktitle = "Proceedings of the 2nd Workshop on Neural Machine Translation and Generation",
    month = jul,
    year = "2018",
    address = "Melbourne, Australia",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/iwcs-25-ingestion/W18-2705/",
    doi = "10.18653/v1/W18-2705",
    pages = "36--44",
    abstract = "Supervised domain adaptation{---}where a large generic corpus and a smaller in-domain corpus are both available for training{---}is a challenge for neural machine translation (NMT). Standard practice is to train a generic model and use it to initialize a second model, then continue training the second model on in-domain data to produce an in-domain model. We add an auxiliary term to the training objective during continued training that minimizes the cross entropy between the in-domain model{'}s output word distribution and that of the out-of-domain model to prevent the model{'}s output from differing too much from the original out-of-domain model. We perform experiments on EMEA (descriptions of medicines) and TED (rehearsed presentations), initialized from a general domain (WMT) model. Our method shows improvements over standard continued training by up to 1.5 BLEU."
}