@inproceedings{si-etal-2025-gateau,
    title = "{GATEAU}: Selecting Influential Samples for Long Context Alignment",
    author = "Si, Shuzheng  and
      Zhao, Haozhe  and
      Chen, Gang  and
      Li, Yunshui  and
      Luo, Kangyang  and
      Lv, Chuancheng  and
      An, Kaikai  and
      Qi, Fanchao  and
      Chang, Baobao  and
      Sun, Maosong",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.375/",
    pages = "7391--7422",
    ISBN = "979-8-89176-332-6",
    abstract = "Aligning large language models to handle instructions with extremely long contexts has yet to be fully investigated. Previous studies have attempted to scale up the available data volume by synthesizing long instruction-following samples, as constructing such a dataset tends to be challenging for annotators. However, a lack of a well-defined strategy for ensuring data quality may introduce low-quality samples and restrict the model{'}s performance. Thus, we propose GATEAU, a novel framework to address the unique challenge of long context alignment by identifying the influential samples enriched with long-range dependency relations. Specifically, GATEAU measures the long-range dependencies from two essential aspects: the difficulty of generating target responses due to the long-range dependencies, and the difficulty of understanding long inputs due to such dependencies. Comprehensive experiments indicate that GATEAU effectively identifies influential samples and the model trained on these selected samples exhibits better instruction-following and long-context understanding capabilities."
}