Bridging the Gap with RedSQL: A Russian Text-to-SQL Benchmark for Domain-Specific Applications

Irina Brodskaya, Elena Tutubalina, Oleg Somov


Abstract
We present the first domain-specific text-to-SQL benchmark in Russian, targeting fields with high operational load where rapid decision-making is critical. The benchmark spans across 9 domains, including healthcare, aviation, and others, and comprises 409 curated query pairs. It is designed to test model generalization under domain shift, introducing challenges such as specialized terminology and complex schema structures. Evaluation of state-of-the-art large language models (LLM) reveals significant performance drop in comparison to open-domain academic benchmarks, highlighting the need for domain-aware approaches in text-to-SQL. The benchmark is available at: https://github.com/BrodskaiaIrina/functional-text2sql-subsets
Anthology ID:
2025.bsnlp-1.9
Volume:
Proceedings of the 10th Workshop on Slavic Natural Language Processing (Slavic NLP 2025)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Jakub Piskorski, Pavel Přibáň, Preslav Nakov, Roman Yangarber, Michal Marcinczuk
Venues:
BSNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
76–83
Language:
URL:
https://preview.aclanthology.org/acl25-workshop-ingestion/2025.bsnlp-1.9/
DOI:
Bibkey:
Cite (ACL):
Irina Brodskaya, Elena Tutubalina, and Oleg Somov. 2025. Bridging the Gap with RedSQL: A Russian Text-to-SQL Benchmark for Domain-Specific Applications. In Proceedings of the 10th Workshop on Slavic Natural Language Processing (Slavic NLP 2025), pages 76–83, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
Bridging the Gap with RedSQL: A Russian Text-to-SQL Benchmark for Domain-Specific Applications (Brodskaya et al., BSNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/acl25-workshop-ingestion/2025.bsnlp-1.9.pdf