@inproceedings{sharma-etal-2025-ttd,
    title = "{TTD}-{SQL}: Tree-Guided Token Decoding for Efficient and Schema-Aware {SQL} Generation",
    author = "Sharma, Chetan  and
      Narayanam, Ramasuri  and
      Pal, Soumyabrata  and
      Yeturu, Kalidas  and
      Saini, Shiv Kumar  and
      Mukherjee, Koyel",
    editor = "Potdar, Saloni  and
      Rojas-Barahona, Lina  and
      Montella, Sebastien",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track",
    month = nov,
    year = "2025",
    address = "Suzhou (China)",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-industry.90/",
    pages = "1287--1298",
    ISBN = "979-8-89176-333-3",
    abstract = "Natural language interfaces (NLIs) democratize data analytics by enabling non-technical users to query relational databases via Text-to-SQL systems. While large language models (LLMs) have achieved state-of-the-art accuracy on benchmarks like Spider and BIRD, two critical challenges persist for real-time deployment: (1) inference latency due to sequential autoregressive decoding (e.g., average inference latency on BIRD (Minidev) is 14.3 seconds per query for Qwen2.5-Coder32B and 22.86 seconds for Llama-70B.), and (2) schema hallucinations (e.g., invalid column references like customer{\_}ids instead of cust{\_}id). (2) schema hallucinations (e.g., Qwen2.5-Coder-32B Instruct generated ... COUNT(users.UserId) ... = users.Id ..., using users.Id correctly in JOIN but hallucinating users.UserId in COUNT). To address these, we propose Tree-Guided Token Decoding (TTD-SQL), a lightweight framework that integrates SQL grammar and database schema constraints into the decoding process without modifying the underlying LLM. TTD precomputes token-level decision trees over SQL keywords, table names, and column identifiers, enabling deterministic ``auto-fill'' transitions for uniquely determined tokens (e.g., ``Song{\_}'' {\textrightarrow} ``ID'') while retaining flexibility for unconstrained reasoning. Across five LLMs (CodeLlama, Phi-4, Qwen2.5, Granite, Llama70B), TTD achieves up to 19.96{\%} token-rate speedups by eliminating redundant forward passes (e.g., CodeLlama: 8.97{\textrightarrow}10.76 tokens/s on Spider) and reduces schema hallucinations by +17.7{\%} in executable-SQL rates (e.g., CodeLlama on BIRD). By bridging rigid parser based methods and flexible LLM generation, TTD offers a practical path toward reliable, high-performance SQL generation in both public benchmarks and enterprise settings."
}