@inproceedings{theologitis-etal-2026-claimdb,
    title = "{C}laim{DB}: A Fact Verification Benchmark over Large Structured Data",
    author = "Theologitis, Michael  and
      Dammu, Preetam Prabhu Srikar  and
      Shah, Chirag  and
      Suciu, Dan",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1589/",
    pages = "34428--34451",
    ISBN = "979-8-89176-390-6",
    abstract = "Real-world fact-checking often involves verifying claims grounded in structured data at scale. Despite substantial progress in fact-verification benchmarks, this setting remains largely underexplored. In this work, we introduce ClaimDB, a fact-verification benchmark where the evidence for claims is derived from compositions of millions of records and multiple tables. ClaimDB consists of 80 unique real-life databases covering a wide range of domains, from governance and healthcare to media, education and the natural sciences. At this scale, verification approaches that rely on ``reading'' the evidence break down, forcing a timely shift toward reasoning in executable programs. We conduct extensive experiments with 30 state-of-the-art proprietary and open-source (below 70B) LLMs and find that more than half score below 55{\%} accuracy. Our analysis also reveals that both closed- and open-source models struggle with abstention {--} the ability to admit that there is no evidence to decide {--} raising doubts about their reliability in high-stakes data analysis tasks. We release the benchmark, code, and the LLM leaderboard at https://claimdb.github.io."
}