OrQA – Open Data Retrieval for Question Answering dataset generation

Giovanni Malaguti, Angelo Mozzillo, Giovanni Simonini


Abstract
We present OrQA, a novel agentic framework to generate large-scale tabular question-answering (TQA) datasets based on real-world open data.Such datasets are needed to overcome the limitations of existing benchmark datasets, which rely on synthetic questions or limited web tables.OrQA employs LLM agents to retrieve related open data tables, generate natural questions, and synthesize executable SQL queries—involving joins, unions, and other non-trivial operations.By leveraging hundreds of GPU hours on four NVIDIA A100, we applied OrQA to Canadian and UK government open data to produce 1,000 question-tables–SQL triples, a representative sample of which has been human‐validated.This open‐source dataset is now publicly available to drive transparency, reproducibility, and progress in table‐based question answering.
Anthology ID:
2025.trl-workshop.14
Volume:
Proceedings of the 4th Table Representation Learning Workshop
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Shuaichen Chang, Madelon Hulsebos, Qian Liu, Wenhu Chen, Huan Sun
Venues:
TRL | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
172–181
Language:
URL:
https://preview.aclanthology.org/acl25-workshop-ingestion/2025.trl-workshop.14/
DOI:
Bibkey:
Cite (ACL):
Giovanni Malaguti, Angelo Mozzillo, and Giovanni Simonini. 2025. OrQA – Open Data Retrieval for Question Answering dataset generation. In Proceedings of the 4th Table Representation Learning Workshop, pages 172–181, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
OrQA – Open Data Retrieval for Question Answering dataset generation (Malaguti et al., TRL 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/acl25-workshop-ingestion/2025.trl-workshop.14.pdf