Summary Factual Inconsistency Detection Based on LLMs Enhanced by Universal Information Extraction

Anguo Li, Lei Yu


Abstract
Automatic text summarization has a potential flaw that affects the factuality of summaries. Recently, Large Language Models (LLMs) have been introduced as detectors for factual inconsistencies in summaries. However, LLM-based methods rely on reasoning capabilities and face challenges in terms of efficiency and explainability. We focus on decoupling LLMs’ information extraction and reasoning capabilities to address prominent challenges, and propose a novel framework, UIEFID (Universal Information Extraction-enhanced Factual Inconsistency Detection). Our idea is to define a self-adaptive structured schema to guide fine-tuned LLMs in extracting unified structured information from documents and summaries, ultimately detecting the origins of inconsistencies in extraction information. The evaluation on 5 open-source models shows that UIEFID not only enhances the detection accuracy on the AGGREFACT benchmark but also significantly reduces redundant reasoning.
Anthology ID:
2025.findings-acl.1305
Volume:
Findings of the Association for Computational Linguistics: ACL 2025
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Wanxiang Che, Joyce Nabende, Ekaterina Shutova, Mohammad Taher Pilehvar
Venues:
Findings | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
25450–25465
Language:
URL:
https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.1305/
DOI:
Bibkey:
Cite (ACL):
Anguo Li and Lei Yu. 2025. Summary Factual Inconsistency Detection Based on LLMs Enhanced by Universal Information Extraction. In Findings of the Association for Computational Linguistics: ACL 2025, pages 25450–25465, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
Summary Factual Inconsistency Detection Based on LLMs Enhanced by Universal Information Extraction (Li & Yu, Findings 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.1305.pdf