IUST_Champs at SemEval-2025 Task 8: Structured Prompting and Retry Policy for Tabular Question Answering

Arshia Hossein Zadeh, Aysa Mayahinia, Nafiseh Ahmadi


Abstract
This paper presents a novel approach to Question Answering over Tabular Data, as part of SemEval-2025 Task 8. Our system generates executable Python code to derive answers directly from structured data, leveraging open-source large language models. Key innovations include structured prompting, semantic column filtering, and a one-time retry mechanism to enhance accuracy and robustness. We evaluate our approach on the DataBench and DataBench_Lite datasets, significantly outperforming the baseline accuracy (26-27%) with our best system achieving 70.49% accuracy on the test set. Ablation studies confirm that few-shot prompting and rule-based type classification are crucial for improved performance. Despite these advancements, challenges remain in handling complex table structures and ambiguous queries. Our findings highlight the effectiveness of code-generation based methods for tabular question answering and provide insights for further research in this area.
Anthology ID:
2025.semeval-1.260
Volume:
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Sara Rosenthal, Aiala Rosá, Debanjan Ghosh, Marcos Zampieri
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2008–2013
Language:
URL:
https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.260/
DOI:
Bibkey:
Cite (ACL):
Arshia Hossein Zadeh, Aysa Mayahinia, and Nafiseh Ahmadi. 2025. IUST_Champs at SemEval-2025 Task 8: Structured Prompting and Retry Policy for Tabular Question Answering. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 2008–2013, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
IUST_Champs at SemEval-2025 Task 8: Structured Prompting and Retry Policy for Tabular Question Answering (Hossein Zadeh et al., SemEval 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.260.pdf