@inproceedings{cai-etal-2025-low,
    title = "Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning",
    author = "Cai, Hongyi  and
      Li, Jie  and
      Rahman, Mohammad Mahdinur  and
      Dong, Wenzhen",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.437/",
    doi = "10.18653/v1/2025.findings-emnlp.437",
    pages = "8233--8240",
    ISBN = "979-8-89176-335-7",
    abstract = "The effectiveness of instruction fine-tuning for Large Language Models is fundamentally constrained by the quality and efficiency of training datasets. This work introduces Low-Confidence Gold (LCG), a novel filtering framework that employs centroid-based clustering and confidence-guided selection for identifying valuable instruction pairs. Through a semi-supervised approach using a lightweight classifier trained on representative samples, LCG curates high-quality subsets while preserving data diversity. Experimental evaluation demonstrates that models fine-tuned on LCG-filtered subsets of 6K samples achieve superior performance compared to existing methods, with substantial improvements on MT-bench and consistent gains across comprehensive evaluation metrics. The framework{'}s efficacy while maintaining model performance establishes a promising result for efficient instruction tuning."
}