OseiBrefo-Liang at SemEval-2025 Task 8 : A Multi-Agent LLM code generation approach for answering Tabular Questions

Emmanuel Osei - Brefo, Huizhi(elly) Liang


Abstract
This paper presents a novel multi-agent framework for automated code generation and execution in tabular question answering. Developed for the SemEval-2025 Task 8, our system utilises a structured, multi-agent approach where distinct agents handle dataset extraction, schema identification, prompt engineering, code generation, execution, and prediction. Unlike traditional methods such as semantic parsing-based SQL generation and transformer-based table models such as TAPAS, our approach leverages a large language model-driven code synthesis pipeline using the DeepSeek API. Our system follows a zero-shot inference approach, which generates Python functions that operate directly on structured data. Through the dynamic extraction of dataset schema and intergration into structured prompts, the model comprehension of tabular structures is enhanced, which leads to more precise and interpretable results. Experimental results demonstrate that our system outperforms existing tabular questioning and answering models, achieving an accuracy of 84.67% on DataBench and 86.02% on DataBench-lite, which significantly surpassed the performances of TAPAS (2.68%) and stable-code-3b-GGUF (27%). The source code used in this paper is available at t https://github.com/oseibrefo/semEval25task8
Anthology ID:
2025.semeval-1.49
Volume:
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Sara Rosenthal, Aiala Rosá, Debanjan Ghosh, Marcos Zampieri
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
343–349
Language:
URL:
https://preview.aclanthology.org/corrections-2025-08/2025.semeval-1.49/
DOI:
Bibkey:
Cite (ACL):
Emmanuel Osei - Brefo and Huizhi(elly) Liang. 2025. OseiBrefo-Liang at SemEval-2025 Task 8 : A Multi-Agent LLM code generation approach for answering Tabular Questions. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 343–349, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
OseiBrefo-Liang at SemEval-2025 Task 8 : A Multi-Agent LLM code generation approach for answering Tabular Questions (Osei - Brefo & Liang, SemEval 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/corrections-2025-08/2025.semeval-1.49.pdf