@inproceedings{xu-etal-2026-rethinking,
    title = "Rethinking Data Mixing from the Perspective of Large Language Models",
    author = "Xu, Yuanjian  and
      Sun, Tianze  and
      Xu, Changwei  and
      Zhao, XinLong  and
      Hao, Jianing  and
      Chen, Ran  and
      Liu, Yang  and
      Xu, Ruijie  and
      Chen, Stephen  and
      Zhang, Guang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 2: Short Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-short.28/",
    pages = "337--349",
    ISBN = "979-8-89176-391-3",
    abstract = "Data mixing strategy is essential for large language model (LLM) training. Empirical evidence shows that inappropriate strategies can significantly reduce generalization. Although recent methods have improved empirical performance, several fundamental questions remain open: what constitutes a domain, whether human and model perceptions of domains are aligned, and how domain weighting influences generalization. We address these questions by establishing formal connections between gradient dynamics and domain distributions, offering a theoretical framework that clarifies the role of domains in training dynamics. Building on this analysis, we introduce DoGraph, a reweighting framework that formulates data scheduling as a graph-constrained optimization problem. Extensive experiments on GPT-2 models of varying scales demonstrate that DoGraph consistently achieves competitive performance."
}