Paul Cayet
2025
Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs
Cong Duy Vu Hoang
|
Gioacchino Tangari
|
Clemence Lanfranchi
|
Dalu Guo
|
Paul Cayet
|
Steve Siu
|
Don Dharmasiri
|
Yuan-Fang Li
|
Long Duong
|
Damien Hilloulin
|
Rhicheek Patra
|
Sungpack Hong
|
Hassan Chafi
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track)
The growing adoption of large language models (LLMs) in business applications has amplified interest in Natural Language to SQL (NL2SQL) solutions, in which there is competing demand for high performance and efficiency. Domain- and customer-specific requirements further complicate the problem. To address this conundrum, we introduce Distill-C, a distilled customization framework tailored for NL2SQL tasks. Distill-C utilizes large teacher LLMs to produce high-quality synthetic data through a robust and scalable pipeline. Finetuning smaller and open-source LLMs on this synthesized data enables them to rival or outperform teacher models an order of magnitude larger. Evaluated on multiple challenging benchmarks, Distill-C achieves an average improvement of 36% in execution accuracy compared to the base models from three distinct LLM families. Additionally, on three internal customer benchmarks, Distill-C demonstrates a 22.6% performance improvement over the base models. Our results demonstrate that Distill-C is an effective, high-performing and generalizable approach for deploying lightweight yet powerful NL2SQL models, delivering exceptional accuracies while maintaining low computational cost.
Search
Fix data
Co-authors
- Hassan Chafi 1
- Don Dharmasiri 1
- Long Duong 1
- Dalu Guo 1
- Damien Hilloulin 1
- show all...