@inproceedings{turek-etal-2025-csiro,
    title = "{CSIRO} {LT} at {S}em{E}val-2025 Task 8: Answering Questions over Tabular Data using {LLM}s",
    author = "Turek, Tomas  and
      Tonni, Shakila Mahjabin  and
      Nguyen, Vincent  and
      Yang, Huichen  and
      Karimi, Sarvnaz",
    editor = "Rosenthal, Sara  and
      Ros{\'a}, Aiala  and
      Ghosh, Debanjan  and
      Zampieri, Marcos",
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.222/",
    pages = "1690--1701",
    ISBN = "979-8-89176-273-2",
    abstract = "Question Answering over large tables is challenging due to the difficulty of reasoning required in linking information from different parts of a table, such as heading and metadata to the values in the table and information needs. We investigate using Large Language Models (LLM) for tabular reasoning, where, given a pair of a table and a question from the DataBench benchmark, the models generate answers. We experiment with three techniques that enables symbolic reasoning through code execution: a direct code prompting (DCP) approach, `DCP{\_}Py', which uses Python, multi-step code (MSC) prompting `MSC{\_}SQL+FS' using SQL and ReAct prompting, `MSR{\_}Py+FS', which combines multi-step reasoning (MSR), few-shot (FS) learning and Python tools. We also conduct an analysis exploring the impact of answer types, data size, and multi-column dependencies on LLMs' answer generation performance, including an assessment of the models' limitations and the underlying challenges of tabular reasoning in LLMs."
}