Abstract
The task of text-to-SQL aims to convert a natural language question into its corresponding SQL query within the context of relational tables. Existing text-to-SQL parsers generate a plausible SQL query for an arbitrary user question, thereby failing to correctly handle problematic user questions. To formalize this problem, we conduct a preliminary study on the observed ambiguous and unanswerable cases in text-to-SQL and summarize them into 6 feature categories. Correspondingly, we identify the causes behind each category and propose requirements for handling ambiguous and unanswerable questions. Following this study, we propose a simple yet effective counterfactual example generation approach that automatically produces ambiguous and unanswerable text-to-SQL examples. Furthermore, we propose a weakly supervised DTE (Detecting-Then-Explaining) model for error detection, localization, and explanation. Experimental results show that our model achieves the best result on both real-world examples and generated examples compared with various baselines. We release our data and code at: https://github.com/wbbeyourself/DTE.- Anthology ID:
- 2023.findings-acl.352
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2023
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5701–5714
- Language:
- URL:
- https://aclanthology.org/2023.findings-acl.352
- DOI:
- 10.18653/v1/2023.findings-acl.352
- Cite (ACL):
- Bing Wang, Yan Gao, Zhoujun Li, and Jian-Guang Lou. 2023. Know What I don’t Know: Handling Ambiguous and Unknown Questions for Text-to-SQL. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5701–5714, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Know What I don’t Know: Handling Ambiguous and Unknown Questions for Text-to-SQL (Wang et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/corrections-2024-04/2023.findings-acl.352.pdf