Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoning

Juraj Vladika, Ivana Hacajova, Florian Matthes


Abstract
Fact verification (FV) aims to assess the veracity of a claim based on relevant evidence. The traditional approach for automated FV includes a three-part pipeline relying on short evidence snippets and encoder-only inference models. More recent approaches leverage the multi-turn nature of LLMs to address FV as a step-by-step problem where questions inquiring additional context are generated and answered until there is enough information to make a decision. This iterative method makes the verification process rational and explainable. While these methods have been tested for encyclopedic claims, exploration on domain-specific and realistic claims is missing. In this work, we apply an iterative FV system on three medical fact-checking datasets and evaluate it with multiple settings, including different LLMs, external web search, and structured reasoning using logic predicates. We demonstrate improvements in the final performance over traditional approaches and the high potential of step-by-step FV systems for domain-specific claims.
Anthology ID:
2025.naacl-short.68
Volume:
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)
Month:
April
Year:
2025
Address:
Albuquerque, New Mexico
Editors:
Luis Chiruzzo, Alan Ritter, Lu Wang
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
805–816
Language:
URL:
https://preview.aclanthology.org/fix-sig-urls/2025.naacl-short.68/
DOI:
Bibkey:
Cite (ACL):
Juraj Vladika, Ivana Hacajova, and Florian Matthes. 2025. Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoning. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 805–816, Albuquerque, New Mexico. Association for Computational Linguistics.
Cite (Informal):
Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoning (Vladika et al., NAACL 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/fix-sig-urls/2025.naacl-short.68.pdf