I2R-NLP at SemEval-2025 Task 8: Question Answering on Tabular Data

Yuze Gao, Bin Chen, Jian Su


Abstract
We present a Large Language Model (LLM) based system for question answering (QA) over tabular data that leverages multi-turn prompting to automatically generate executable Pandas functions. Our framework decomposes the problem into three key steps: (1) Answer Type Identification, where the system identifies the expected format of the response (e.g., boolean, number, category); (2) Pandas Function Generation, which generates a corresponding Pandas function using table metadata and in-context examples, and (3) Error Correction and Regeneration, where iteratively refining the function based on error feedback from executions. Evaluations on the SemEval-2025 Task 8 Tabular QA benchmark (Grijalba et al., 2024) demonstrate that our multi-turn approach significantly outperforms single-turn prompting models in exact match accuracy by 7.3%. The proposed system not only improves code generation robustness but also paves the way for enhanced and adaptability in table-QA reasoning tasks. Our implementation is available at https://github.com/Gyyz/Question_Answering-over-Tabular-Data.
Anthology ID:
2025.semeval-1.14
Volume:
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Sara Rosenthal, Aiala Rosá, Debanjan Ghosh, Marcos Zampieri
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
90–101
Language:
URL:
https://preview.aclanthology.org/corrections-2025-08/2025.semeval-1.14/
DOI:
Bibkey:
Cite (ACL):
Yuze Gao, Bin Chen, and Jian Su. 2025. I2R-NLP at SemEval-2025 Task 8: Question Answering on Tabular Data. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 90–101, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
I2R-NLP at SemEval-2025 Task 8: Question Answering on Tabular Data (Gao et al., SemEval 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/corrections-2025-08/2025.semeval-1.14.pdf