@inproceedings{zhu-etal-2025-text2sql,
    title = "{T}ext2{S}ql: Pure Fine-Tuning and Pure Knowledge Distillation",
    author = "Zhu, Gao yu  and
      Shao, Wei  and
      Zhu, Xichou  and
      Yu, Lei  and
      Guo, Jiafeng  and
      Cheng, Xueqi",
    editor = "Chen, Weizhu  and
      Yang, Yi  and
      Kachuee, Mohammad  and
      Fu, Xue-Yong",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.naacl-industry.5/",
    pages = "54--61",
    ISBN = "979-8-89176-194-0",
    abstract = "Text2Sql is a task that converts natural language questions into SQL queries. In previous research on LLM fine-tuning, researchers typically input both the entire database schema and the natural language question into the model. This approach has two issues: 1) the model{'}s context is limited when dealing with a large number of database tables; 2) the question is often related to only a few tables, leading to excessive irrelevant information that distracts the model. To address these issues, we employed pure fine-tuning strategy to reduce redundancy. The model fine-tuned with pure prompts, using prompts that are only 53{\%} of the baseline length, outperforms the baseline (fine-tuned with all tables in the prompt) by 8.2{\%} and 8.6{\%} in Test-suite accuracy (TS) and exact-set-match accuracy (EM), respectively, on the Spider dev set. Under the most refined Spider dev set of prompts, the model achieves TS and EM scores of 73.5{\%} and 75.4{\%}, respectively, approaching state-of-the-art (SOTA) levels. To leverage the capabilities of the model with pure prompts, we applied pure knowledge distillation strategy to transfer its abilities. The distilled student model achieved a 1.9{\%} improvement in TS, while the teacher model{'}s prompt length was only 23{\%} of that of the student model."
}