@inproceedings{sun-etal-2026-rethinking,
    title = "Rethinking Text-to-{SQL}: Dynamic Multi-turn {SQL} Interaction for Real-world Database Exploration",
    author = "Sun, Linzhuang  and
      Guo, Tianyu  and
      Liang, Hao  and
      Liu, Ruitong  and
      Li, Yuying  and
      Cai, Qifeng  and
      Wei, Jingxuan  and
      Wu, Yuchen  and
      Yu, Bihui  and
      Zhang, Xiangxiang  and
      Zhang, Wentao  and
      Cui, Bin",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1654/",
    pages = "33047--33069",
    ISBN = "979-8-89176-395-1",
    abstract = "Recent advancements in Large Language Models (LLMs) have revolutionized Text-to-SQL parsing, achieving remarkable success in static, single-turn query generation. However, a significant disparity remains between these academic benchmarks and real-world utility. In practical applications, such as financial auditing or business analytics, user intents are rarely static; they evolve dynamically through iterative refinement, necessitating not just information retrieval (SELECT) but continuous state manipulation (INSERT, UPDATE, DELETE). To bridge this gap, we introduce DySQL-Bench, a novel benchmark designed to rigorously evaluate LLMs within a dynamic interaction framework. Unlike varying manual curation efforts, DySQL-Bench employs a two-stage automated synthesis pipeline: transforming raw relational schemas into hierarchical logic trees to generate user-database interactions, followed by a rigorous verify-and-refine protocol that ensures 100{\%} distinct correctness via human expert validation. We further propose an interactive evaluation environment simulating a triadic workflow involving an LLM-simulated user, the agent under test, and an executable database system. Spanning 13 diverse domains with 1,072 complex tasks, our experiments reveal that current powerful models struggle in this realistic setting. Notably, GPT-4o achieves only 58.34{\%} overall accuracy and a meager 23.81{\%} on the strict Pass{\textasciicircum}5 metric, highlighting the substantial challenges DySQL-Bench poses for the future of database agents."
}