Yifan Wang
Other people with similar names: Yifan Wang, Yifan Wang, Yifan Wang, Yifan Wang, Yifan Wang, Yifan Wang
Unverified author pages with similar names: Yifan Wang
2026
TiKMiX: Efficient Semi-Dynamic Data Mixture via Data Influence for LLM Pre-training
Yifan Wang | Binbinliu | Fengze Liu | Yuanfan Guo | Jiyao Deng | Xuecheng Wu | Weidong Zhou | Xiaohuan Zhou | Taifeng Wang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yifan Wang | Binbinliu | Fengze Liu | Yuanfan Guo | Jiyao Deng | Xuecheng Wu | Weidong Zhou | Xiaohuan Zhou | Taifeng Wang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
The data mixture used in the pre-training of a language model is a cornerstone of its final performance. Static data mixing strategies in Large Language Model (LLM) pre-training are often suboptimal as they fail to adapt to the model’s evolving learning states. Conversely, fully online dynamic updates, while adaptive, incur prohibitive computational costs. To bridge this gap, we propose TiKMiX, an efficient semi-dynamic data mixing framework. Our approach is grounded in a key observation of influence ranking invariance: the relative importance of data domains exhibits strong temporal stability over long training intervals. Leveraging this insight, we propose Group Influence, an efficient approach for quantifying domain impact, and formulate data mixing as a periodic, low-overhead influence maximization problem. Compared with REGMIX, the proposed method reduces computational overhead by 80% and achieves an average performance gain of 2% across nine downstream benchmarks, thereby effectively mitigating data under-digestion.