BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra

Parker Glenn, Parag Dakle, Liang Wang, Preethi Raghavan

Abstract
Many existing end-to-end systems for hybrid question answering tasks can often be boiled down to a “prompt-and-pray” paradigm, where the user has limited control and insight into the intermediate reasoning steps used to achieve the final result. Additionally, due to the context size limitation of many transformer-based LLMs, it is often not reasonable to expect that the full structured and unstructured context will fit into a given prompt in a zero-shot setting, let alone a few-shot setting. We introduce BlendSQL, a superset of SQLite to act as a unified dialect for orchestrating reasoning across both unstructured and structured data. For hybrid question answering tasks involving multi-hop reasoning, we encode the full decomposed reasoning roadmap into a single interpretable BlendSQL query. Notably, we show that BlendSQL can scale to massive datasets and improve the performance of end-to-end systems while using 35% fewer tokens. Our code is available and installable as a package at https://github.com/parkervg/blendsql.
Anthology ID:
2024.findings-acl.25
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
453–466
Language:
URL:
https://aclanthology.org/2024.findings-acl.25
DOI:
Bibkey:
Cite (ACL):
Parker Glenn, Parag Dakle, Liang Wang, and Preethi Raghavan. 2024. BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra. In Findings of the Association for Computational Linguistics ACL 2024, pages 453–466, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra (Glenn et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.findings-acl.25.pdf
PDF Search