@inproceedings{luo-etal-2025-beyond,
    title = "Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation",
    author = "Luo, Yingfeng  and
      Zheng, Tong  and
      Mu, Yongyu  and
      Li, Bei  and
      Zhang, Qinghong  and
      Gao, Yongqi  and
      Xu, Ziqiang  and
      Feng, Peinan  and
      Liu, Xiaoqian  and
      Xiao, Tong  and
      Zhu, JingBo",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.490/",
    pages = "9399--9431",
    ISBN = "979-8-89176-256-5",
    abstract = "The field of neural machine translation (NMT) has changed with the advent of large language models (LLMs). Much of the recent emphasis in natural language processing (NLP) has been on modeling machine translation and many other problems using a single pre-trained Transformer decoder, while encoder-decoder architectures, which were the standard in earlier NMT models, have received relatively less attention. In this paper, we explore translation models that are universal, efficient, and easy to optimize, by marrying the world of LLMs with the world of NMT. We apply LLMs to NMT encoding and leave the NMT decoder unchanged. We also develop methods for adapting LLMs to work better with the NMT decoder. Furthermore, we construct a new dataset involving multiple tasks to assess how well the machine translation system generalizes across various tasks. Evaluations on the WMT and our datasets show that results using our method match or surpass a range of baselines in terms of translation quality, but achieve $2.4 \sim 6.5 \times$ inference speedups and a 75{\%} reduction in the memory footprint of the KV cache. It also demonstrates strong generalization across a variety of translation-related tasks."
}