@inproceedings{wang-etal-2026-tikmix,
    title = "{T}i{KM}i{X}: Efficient Semi-Dynamic Data Mixture via Data Influence for {LLM} Pre-training",
    author = "Wang, Yifan  and
      Binbinliu  and
      Liu, Fengze  and
      Guo, Yuanfan  and
      Deng, Jiyao  and
      Wu, Xuecheng  and
      Zhou, Weidong  and
      Zhou, Xiaohuan  and
      Wang, Taifeng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.261/",
    pages = "5777--5793",
    ISBN = "979-8-89176-390-6",
    abstract = "The data mixture used in the pre-training of a language model is a cornerstone of its final performance. Static data mixing strategies in Large Language Model (LLM) pre-training are often suboptimal as they fail to adapt to the model{'}s evolving learning states. Conversely, fully online dynamic updates, while adaptive, incur prohibitive computational costs. To bridge this gap, we propose TiKMiX, an efficient semi-dynamic data mixing framework. Our approach is grounded in a key observation of influence ranking invariance: the relative importance of data domains exhibits strong temporal stability over long training intervals. Leveraging this insight, we propose Group Influence, an efficient approach for quantifying domain impact, and formulate data mixing as a periodic, low-overhead influence maximization problem. Compared with REGMIX, the proposed method reduces computational overhead by 80{\%} and achieves an average performance gain of 2{\%} across nine downstream benchmarks, thereby effectively mitigating data under-digestion."
}