@inproceedings{chunkang-etal-2024-pattern,
    title = "Pattern Shifting or Knowledge Losing? A Forgetting Perspective for Understanding the Effect of Instruction Fine-Tuning",
    author = "Zhang, Chunkang  and
      Cao, Boxi  and
      Lu, Yaojie  and
      Lin, Hongyu  and
      Cao, Liu  and
      Zeng, Ke  and
      Wan, Guanglu  and
      Cai, Xunliang  and
      Han, Xianpei  and
      Sun, Le",
    editor = "Maosong, Sun  and
      Jiye, Liang  and
      Xianpei, Han  and
      Zhiyuan, Liu  and
      Yulan, He",
    booktitle = "Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 1: Main Conference)",
    month = jul,
    year = "2024",
    address = "Taiyuan, China",
    publisher = "Chinese Information Processing Society of China",
    url = "https://preview.aclanthology.org/author-degibert/2024.ccl-1.106/",
    pages = "1381--1394",
    language = "eng",
    abstract = "``Instruction Fine-Tuning(IFT) emerges as an essential step of training large language models torobustly carry out tasks of interest. However, there lacks a systematic investigation about theunderlying mechanisms of instruction fine-tuning, particularly on the forgetting phenomenonafter IFT, known as alignment tax. Therefore, to understand the mechanism of IFT from theforgetting perspective, we investigate the alternation of the text pattern and knowledge withinmodels throughout the entire IFT process. Specifically, we restore fine-tuned models to their baseversion by training them on the data sharing a similar distribution with the pre-training corpusand compare their results Our experiment indicates that there is a stage transition of forgettingduring IFT process: (1) Pseudo Forgetting: in this stage, models mainly shift their familiar textpattern away from pre-training data format while the world knowledge is preserved. Consequently,models will recover to their original performance when they are restored to the base version. (2)Actual Forgetting: in this stage, models forget the acquired knowledge as well. Therefore, theyfail to reach the original performance even if they are restored to the base version.''"
}