A Logic-Based Approach to Hallucinations in Data-to-Text NLG: Experiments with Human and LLM Annotators

Eduardo Calò, Saad Mahamood, Albert Gatt, Kees Van Deemter


Abstract
Hallucinations are a persistent challenge in natural language generation, including data-to-text. van Deemter (2024) introduced a framework based on the relation of logical consequence ("follows from"), which divides all data-to-text hallucinations into seven disjoint categories. We examine whether human annotators and large language models are able to apply the framework, in two data-to-text domains. Results suggest that the framework is applicable, although there are significant domain-dependent variations, as well as discrepancies between human and model judgments. We also uncover several issues that should inform future work on hallucination.
Anthology ID:
2026.starsem-conference.3
Volume:
Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Saif M. Mohammad, Nedjma Ousidhoum
Venues:
*SEM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
28–62
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.starsem-conference.3/
DOI:
Bibkey:
Cite (ACL):
Eduardo Calò, Saad Mahamood, Albert Gatt, and Kees Van Deemter. 2026. A Logic-Based Approach to Hallucinations in Data-to-Text NLG: Experiments with Human and LLM Annotators. In Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026), pages 28–62, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
A Logic-Based Approach to Hallucinations in Data-to-Text NLG: Experiments with Human and LLM Annotators (Calò et al., *SEM 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.starsem-conference.3.pdf