@inproceedings{zhao-etal-2025-beyond-similarity,
    title = "Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection",
    author = "Zhao, Yang  and
      Du, Li  and
      Ding, Xiao  and
      Ouyang, Yangou  and
      Wang, Hepeng  and
      Xiong, Kai  and
      Gao, Jinglong  and
      Sun, Zhouhao  and
      Xu, Dongliang  and
      Yang, Qing  and
      Li, Dongchen  and
      Qin, Bing  and
      Liu, Ting",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1189/",
    pages = "24391--24404",
    ISBN = "979-8-89176-251-0",
    abstract = "Large language models (LLMs) have shown great potential across various industries due to their remarkable ability to generalize through instruction tuning. However, the limited availability of domain-specific data significantly hampers their performance on specialized tasks. While existing methods primarily focus on selecting training data from general datasets that are similar to the target domain, they often fail to consider the joint distribution of instructions, resulting in inefficient learning and suboptimal knowledge transfer. To address these challenges, we introduce **G2IS** (**G**radient-based **G**raph **I**nstruction **S**election), a novel method that constructs a mixed gradient-based instruction graph to capture the joint distribution and interdependencies among instructions. By accounting for the relationships between instructions, G2IS improves domain adaptation efficiency. Additionally, we propose a gradient walk algorithm to refine the data selection process, enhancing both training effectiveness and efficiency. Our experiments demonstrate that G2IS outperforms traditional methods across various domain adaptation tasks, yielding significant performance gains, particularly in complex, data-scarce scenarios. These results underscore the potential of G2IS in advancing the development of large, domain-specific models."
}