@inproceedings{zhang-etal-2023-machine,
    title = "Machine Translation with Large Language Models: Prompting, Few-shot Learning, and Fine-tuning with {QL}o{RA}",
    author = "Zhang, Xuan  and
      Rajabi, Navid  and
      Duh, Kevin  and
      Koehn, Philipp",
    editor = "Koehn, Philipp  and
      Haddow, Barry  and
      Kocmi, Tom  and
      Monz, Christof",
    booktitle = "Proceedings of the Eighth Conference on Machine Translation",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2023.wmt-1.43/",
    doi = "10.18653/v1/2023.wmt-1.43",
    pages = "468--481",
    abstract = "While large language models have made remarkable advancements in natural language generation, their potential in machine translation, especially when fine-tuned, remains under-explored. In our study, we conduct comprehensive experiments, evaluating 15 publicly available language models on machine translation tasks. We compare the performance across three methodologies: zero-shot prompting, few-shot learning, and fine-tuning. Central to our approach is the use of QLoRA, an efficient fine-tuning method. On French-English, QLoRA fine-tuning outperforms both few-shot learning and models trained from scratch. This superiority is highlighted in both sentence-level and document-level translations, with a significant BLEU score improvement of 28.93 over the prompting method. Impressively, with QLoRA, the enhanced performance is achieved by fine-tuning a mere 0.77{\%} of the model{'}s parameters."
}