@inproceedings{ye-etal-2025-disentangling,
    title = "Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning",
    author = "Ye, Ziang  and
      Zhang, Zhenru  and
      Zhang, Yang  and
      Ma, Jianxin  and
      Lin, Junyang  and
      Feng, Fuli",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.findings-acl.1078/",
    pages = "20939--20957",
    ISBN = "979-8-89176-256-5",
    abstract = "When using agent-task datasets to enhance agent capabilities for Large Language Models (LLMs), current methodologies often treat all tokens within a sample equally. However, we argue that tokens serving different roles{---}specifically, reasoning tokens versus boilerplate tokens (e.g., those governing output format){---}differ significantly in importance and learning complexity, necessitating their disentanglement and distinct treatment. To address this, we propose a novel Shuffle-Aware Discriminator (SHAD) for adaptive token discrimination. SHAD classifies tokens by exploiting predictability differences observed after shuffling input-output combinations across samples: boilerplate tokens, due to their repetitive nature among samples, maintain predictability, whereas reasoning tokens do not. Using SHAD, we propose the Reasoning-highlighted Fine-Tuning (RFT) method, which adaptively emphasizes reasoning tokens during fine-tuning, yielding notable performance gains over common Supervised Fine-Tuning (SFT)."
}