@inproceedings{lu-etal-2020-mixed,
    title = "A Mixed Learning Objective for Neural Machine Translation",
    author = "Lu, Wenjie  and
      Zhou, Leiying  and
      Liu, Gongshen  and
      Zhang, Quanhai",
    editor = "Sun, Maosong  and
      Li, Sujian  and
      Zhang, Yue  and
      Liu, Yang",
    booktitle = "Proceedings of the 19th Chinese National Conference on Computational Linguistics",
    month = oct,
    year = "2020",
    address = "Haikou, China",
    publisher = "Chinese Information Processing Society of China",
    url = "https://preview.aclanthology.org/fix-sig-urls/2020.ccl-1.90/",
    pages = "974--983",
    language = "eng",
    abstract = "Evaluation discrepancy and overcorrection phenomenon are two common problems in neural machine translation (NMT). NMT models are generally trained with word-level learning objective, but evaluated by sentence-level metrics. Moreover, the cross-entropy loss function discourages model to generate synonymous predictions and overcorrect them to ground truth words. To address these two drawbacks, we adopt multi-task learning and propose a mixed learning objective (MLO) which combines the strength of word-level and sentence-level evaluation without modifying model structure. At word-level, it calculates semantic similarity between predicted and ground truth words. At sentence-level, it computes probabilistic n-gram matching scores of generated translations. We also combine a loss-sensitive scheduled sampling decoding strategy with MLO to explore its extensibility. Experimental results on IWSLT 2016 German-English and WMT 2019 English-Chinese datasets demonstrate that our methodology can significantly promote translation quality. The ablation study shows that both word-level and sentence-level learning objectives can improve BLEU scores. Furthermore, MLO is consistent with state-of-the-art scheduled sampling methods and can achieve further promotion."
}