PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL
Ruilin Luo, Liyuan Wang, Binghuai Lin, Zicheng Lin, Yujiu Yang
Abstract
Large Language Models (LLMs) have emerged as powerful tools for Text-to-SQL tasks, exhibiting remarkable reasoning capabilities. Different from tasks such as math word problem and commonsense reasoning, SQL solutions have a relatively fixed pattern. This facilitates the investigation of whether LLMs can benefit from categorical thinking, mirroring how humans acquire knowledge through inductive reasoning based on comparable examples. In this study, we propose that employing query group partitioning allows LLMs to focus on learning the thought processes specific to a single problem type, consequently enhancing their reasoning abilities across diverse difficulty levels and problem categories. Our experiments reveal that multiple advanced LLMs, when equipped with PTD-SQL, can either surpass or match previous state-of-the-art (SOTA) methods on the Spider and BIRD datasets. Intriguingly, models with varying initial performances have exhibited significant improvements mainly at the boundary of their capabilities after targeted drilling, suggesting a parallel with human progress. Code is available at https://github.com/lrlbbzl/PTD-SQL.- Anthology ID:
- 2024.emnlp-main.221
- Volume:
- Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3767–3799
- Language:
- URL:
- https://aclanthology.org/2024.emnlp-main.221
- DOI:
- 10.18653/v1/2024.emnlp-main.221
- Cite (ACL):
- Ruilin Luo, Liyuan Wang, Binghuai Lin, Zicheng Lin, and Yujiu Yang. 2024. PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3767–3799, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL (Luo et al., EMNLP 2024)
- PDF:
- https://preview.aclanthology.org/landing_page/2024.emnlp-main.221.pdf