@inproceedings{ashury-tahan-etal-2026-mighty,
    title = "The Mighty {T}o{RR}: A Benchmark for Table Reasoning and Robustness in {LLM}s",
    author = "Ashury-Tahan, Shir  and
      Mai, Yifan  and
      C, Rajmohan  and
      Gera, Ariel  and
      Perlitz, Yotam  and
      Yehudai, Asaf  and
      Bandel, Elron  and
      Choshen, Leshem  and
      Shnarch, Eyal  and
      Liang, Percy  and
      Shmueli-Scheuer, Michal",
    editor = "Gupta, Vivek  and
      Ding, Kaize  and
      Kokel, Harsha  and
      Zhao, Yue  and
      Agarwal, Amit  and
      Wang, Yu  and
      Glass, Michael  and
      Zhang, Yu  and
      Srinivas, Kavitha  and
      Chen, Xiusi  and
      Hassanzadeh, Oktie  and
      Zhu, Qi  and
      Chang, Shuaichen  and
      Luo, Yuan",
    booktitle = "Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the {LLM} Era ({SURG}e{LLM} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.surgellm-1.2/",
    pages = "16--51",
    ISBN = "979-8-89176-406-4",
    abstract = "Despite its real-world significance, model performance on tabular data remains underexplored, leaving uncertainty about which model to rely on and which prompt configuration to adopt. To address this gap, we create ToRR, a benchmark for Table Reasoning and Robustness, measuring model performance and robustness on table-related tasks. The benchmark includes 10 datasets that cover different types of table reasoning capabilities across varied domains. ToRR goes beyond model performance rankings, and is designed to reflect whether models can handle tabular data consistently and robustly, across a variety of common table representation formats. We present a leaderboard as well as comprehensive analyses of the results of leading models over ToRR. Our results reveal a striking pattern of brittle model behavior, where even strong models are unable to perform robustly on tabular data tasks. We further find that no single table format consistently yields superior performance. However, evaluating models across multiple formats is essential for a reliable assessment of their capabilities. Moreover, we show that the reliability boost from testing multiple prompts can be equivalent to adding more test examples. Overall, our findings show that reasoning over table tasks remains a significant challenge. The leaderboard, data and code are publicly available."
}