@inproceedings{hu-etal-2026-fine,
    title = "Fine-Grained Data Ordering Improves Fine-Tuning for Large Language Models",
    author = "Hu, Xiaomeng  and
      Tang, Yixuan  and
      Li, Haoze  and
      Chen, Hao  and
      Zhang, Qi  and
      Shen, Zhanming  and
      Zhang, Yiming  and
      Wang, Haobo  and
      Zhao, Junbo",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1021/",
    pages = "20406--20418",
    ISBN = "979-8-89176-395-1",
    abstract = "With the rapid progress of large language models (LLMs), aligning a general-purpose model with downstream tasks through fine-tuning has become a central research focus. Selecting only high-quality examples for training has been shown to be one of the most effective ways to improve fine-tuning performance. However, prior work concentrates almost exclusively on data preprocessing: filtering and cleaning data before training begins. While the order and composition of training data during training have received little fine-grained attention. To fill this gap, our work proposed Fine-Grained Order Fine-Tuning, a fine-grained scheduling method of data order in epochs. Drawing on curriculum-learning principles, FOT defines data difficulty based on the relevance between the data and the model, and then performs dynamic scheduling of the training order in each epoch according to the difficulty. On both large-scale continued pre-training and small-scale supervised fine-tuning experiments, FOT has achieved an average 2.4{\%} improvement over baselines. Our study offers a new perspective on data governance in the fine-tuning phase."
}