A Discourse-based Tool Series for Logical Validation of LLMs

Boris Galitsky, Dmitry Ilvovsky


Abstract
Large Language Models (LLMs) frequently produce fluent but unverifiable reasoning, resulting in potential hallucinations and faulty inferences. This study proposes a logic programming - based verification framework ValidLogic4LLM in which the reasoning expressed by an LLM is transformed into a logic program (LP), probabilistic LP, defeasible LP and abductive LP representing world knowledge and a given problem description—such as a patient health complaint. The LP formed by an LLM is executed within a symbolic reasoning engine, and the resulting inferences are compared to the LLM’s natural-language conclusions. The strength or probability of facts, clauses and arguments is computed based on discourse structure of text expressing these facts or arguments. Divergence between symbolic and neural reasoning outcomes indicates possible hallucination or inconsistency in the model’s internal logic.
Anthology ID:
2026.lrec-main.174
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
2220–2231
Language:
URL:
https://preview.aclanthology.org/ingest-lrec/2026.lrec-main.174/
DOI:
Bibkey:
Cite (ACL):
Boris Galitsky and Dmitry Ilvovsky. 2026. A Discourse-based Tool Series for Logical Validation of LLMs. International Conference on Language Resources and Evaluation, main:2220–2231.
Cite (Informal):
A Discourse-based Tool Series for Logical Validation of LLMs (Galitsky & Ilvovsky, LREC 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-lrec/2026.lrec-main.174.pdf