Face the Facts! Evaluating RAG-based Pipelines for Professional Fact-Checking

Daniel Russo, Stefano Menini, Jacopo Staiano, Marco Guerini


Abstract
Natural Language Processing and Generation systems have recently shown the potential to complement and streamline the costly and time-consuming job of professional fact-checkers. In this work, we lift several constraints of current state-of-the-art pipelines for automated fact-checking based on the Retrieval-Augmented Generation (RAG) paradigm. Our goal is to benchmark, following professional fact-checking practices, RAG-based methods for the generation of verdicts - i.e., short texts discussing the veracity of a claim - evaluating them on stylistically complex claims and heterogeneous, yet reliable, knowledge bases. Our findings show a complex landscape, where, for example, LLM-based retrievers outperform other retrieval techniques, though they still struggle with heterogeneous knowledge bases; larger models excel in verdict faithfulness, while smaller models provide better context adherence, with human evaluations favouring zero-shot and one-shot approaches for informativeness, and fine-tuned models for emotional alignment.
Anthology ID:
2025.inlg-main.50
Volume:
Proceedings of the 18th International Natural Language Generation Conference
Month:
October
Year:
2025
Address:
Hanoi, Vietnam
Editors:
Lucie Flek, Shashi Narayan, Lê Hồng Phương, Jiahuan Pei
Venue:
INLG
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
846–865
Language:
URL:
https://preview.aclanthology.org/ingest-luhme/2025.inlg-main.50/
DOI:
Bibkey:
Cite (ACL):
Daniel Russo, Stefano Menini, Jacopo Staiano, and Marco Guerini. 2025. Face the Facts! Evaluating RAG-based Pipelines for Professional Fact-Checking. In Proceedings of the 18th International Natural Language Generation Conference, pages 846–865, Hanoi, Vietnam. Association for Computational Linguistics.
Cite (Informal):
Face the Facts! Evaluating RAG-based Pipelines for Professional Fact-Checking (Russo et al., INLG 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-luhme/2025.inlg-main.50.pdf