@inproceedings{ren-etal-2024-learn,
    title = "{I} Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with {LLM}-Generated Responses",
    author = "Ren, Xuan  and
      Wu, Biao  and
      Liu, Lingqiao",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2024.emnlp-main.571/",
    doi = "10.18653/v1/2024.emnlp-main.571",
    pages = "10225--10245",
    abstract = "This paper explores an intriguing observation: fine-tuning a large language model (LLM) with responses generated by a LLM often yields better results than using responses generated by humans, particularly in reasoning tasks. We conduct an in-depth investigation to understand why this occurs. Contrary to the common belief that these instances is due to the more detailed nature of LLM-generated content, our study identifies another contributing factor: an LLM is inherently more {\textquotedblleft}familiar{\textquotedblright} with LLM generated responses. This familiarity is evidenced by lower perplexity before fine-tuning. We design a series of experiments to understand the impact of the {\textquotedblleft}familiarity{\textquotedblright} and our conclusion reveals that this {\textquotedblleft}familiarity{\textquotedblright} significantly impacts learning performance. Training with LLM-generated responses not only enhances performance but also helps maintain the model`s capabilities in other reasoning tasks after fine-tuning on a specific task."
}