@inproceedings{zhao-etal-2025-cds,
    title = "{CDS}: Data Synthesis Method Guided by Cognitive Diagnosis Theory",
    author = "Zhao, Haokun  and
      Han, Jinyi  and
      Liang, Jiaqing  and
      Xiao, Yanghua  and
      Meng, Xiaojun  and
      Wei, Jiansheng",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.439/",
    pages = "8370--8393",
    ISBN = "979-8-89176-256-5",
    abstract = "Large Language Models (LLMs) have achieved significant advancements, but the increasing complexity of tasks and higher performance demands highlight the need for continuous improvement. Some approaches utilize synthetic data generated by advanced LLMs based on evaluation results to train models. However, conventional evaluation methods fail to provide detailed, fine-grained profiles of LLMs, limiting their guidance for data synthesis. In this paper, we introduce the **Cognitive Diagnostic Synthesis** (CDS) method, which incorporates a diagnostic process inspired by **Cognitive Diagnosis Theory** (CDT) to refine evaluation results and characterize model profiles at the knowledge component level. Based on these diagnostics, we propose two diagnosis-synthesis strategies for weakness-targeted data synthesis. Additionally, we present an enhanced data augmentation and selection pipeline to improve the quality and diversity of synthesized data. Our experiments with several open-source models show significant improvements across multiple benchmarks, achieving up to 6.00{\%} improvement in code generation, 13.10{\%} in mathematical reasoning, and 5.43{\%} in academic exams. Code and data are available on GitHub https://anonymous.4open.science/r/cds-04D1."
}