@inproceedings{yang-etal-2026-comptab,
    title = "{C}omp{T}ab: A Comprehensive Benchmark for Real-World {T}able{QA} with Complex Reasoning and Irregular Tables",
    author = "Yang, Zhen  and
      Du, Wei  and
      Wang, Jie  and
      Zhou, Wenze  and
      Meng, Xiangfeng  and
      Wang, Zhengyang  and
      Sun, Suping  and
      Du, Ziwei  and
      Zou, Haodong  and
      Chen, Jie  and
      Liu, Yongbin  and
      Tan, Shicheng  and
      Ying, Jiahao  and
      Zhao, Shu",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1279/",
    pages = "27761--27774",
    ISBN = "979-8-89176-390-6",
    abstract = "Recent progress in Large Language Model (LLM) based Table Question Answering (TableQA) has demonstrated strong performance on standard benchmarks. However, existing benchmarks mainly focus on well-structured tables and fail to reflect the irregular structures and complex reasoning commonly encountered in real-world scenarios. We propose CompTab, a benchmark designed to evaluate TableQA under complex reasoning and irregular table conditions. CompTab covers six representative types, including semantic ambiguity, multi-hop reasoning, transposed tables, merged cells, missing values, and outliers. It is constructed from real-world seed tables across multiple domains using controlled LLM based generation and human verification to ensure realism and diversity. In addition, to improve the generalization of LLMs under complex and irregular table settings, we propose a two-stage training framework that progressively aligns models with textual reasoning and executable decision signals, instantiated as CompTabLLM. Evaluations on 38 representative LLMs and CompTabLLM show clear limitations of existing LLMs under realistic conditions, while the proposed framework improves generalization. CompTab thus provides a challenging benchmark for advancing TableQA in real-world."
}