@inproceedings{n-elnokrashy-etal-2022-language,
    title = "Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation",
    author = "ElNokrashy, Muhammad  and
      Hendy, Amr  and
      Maher, Mohamed  and
      Afify, Mohamed  and
      Hassan Awadalla, Hany",
    editor = "Duh, Kevin  and
      Guzm{\'a}n, Francisco",
    booktitle = "Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)",
    month = sep,
    year = "2022",
    address = "Orlando, USA",
    publisher = "Association for Machine Translation in the Americas",
    url = "https://preview.aclanthology.org/fix-sig-urls/2022.amta-research.6/",
    pages = "70--82",
    abstract = "This paper proposes a simple yet effective method to improve direct (X-to-Y) translation for both cases: zero-shot and when direct data is available. We modify the input tokens at both the encoder and decoder to include signals for the source and target languages. We show a performance gain when training from scratch, or finetuning a pretrained model with the pro- posed setup. In the experiments, our method shows nearly 10.0 BLEU points gain on in-house datasets depending on the checkpoint selection criteria. In a WMT evaluation campaign, From- English performance improves by 4.17 and 2.87 BLEU points, in the zero-shot setting, and when direct data is available for training, respectively. While X-to-Y improves by 1.29 BLEU over the zero-shot baseline, and 0.44 over the many-to-many baseline. In the low-resource setting, we see a 1.5 {\ensuremath{\sim}} 1.7 point improvement when finetuning on X-to-Y domain data."
}