@inproceedings{liangyu-etal-2025-fire,
    title = "{FIRE}: Flexible Integration of Data Quality Ratings for Effective Pretraining",
    author = "Liangyu, Xu  and
      Zhang, Xuemiao  and
      Duan, Feiyu  and
      Wang, Sirui  and
      Weng, Rongxiang  and
      Wang, Jingang  and
      Cai, Xunliang",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.735/",
    pages = "14532--14552",
    ISBN = "979-8-89176-332-6",
    abstract = "Selecting high-quality data can improve the pretraining efficiency of large language models (LLMs). Existing methods generally rely on heuristic techniques or single quality signals, limiting their ability to evaluate data quality comprehensively. In this work, we propose FIRE, a flexible and scalable framework for integrating multiple data quality raters, which allows for a comprehensive assessment of data quality across various dimensions. FIRE aligns multiple quality signals into a unified space, and integrates diverse data quality raters to provide a comprehensive quality signal for each data point. Further, we introduce a progressive data selection scheme based on FIRE that iteratively refines the selection of high-quality data points. Extensive experiments show that FIRE outperforms other data selection methods and significantly boosts pretrained model performance across a wide range of downstream tasks, while requiring less than 37.5{\%} tokens needed by the $\textit{Random}$ baseline to reach the target performance."
}