InFact: A Strong Baseline for Automated Fact-Checking
Mark Rothermel, Tobias Braun, Marcus Rohrbach, Anna Rohrbach
Abstract
The spread of disinformation poses a global threat to democratic societies, necessitating robust and scalable Automated Fact-Checking (AFC) systems. The AVeriTeC Shared Task Challenge 2024 offers a realistic benchmark for text-based fact-checking methods. This paper presents Information-Retrieving Fact-Checker (InFact), an LLM-based approach that breaks down the task of claim verification into a 6-stage process, including evidence retrieval. When using GPT-4o as the backbone, InFact achieves an AVeriTeC score of 63% on the test set, outperforming all other 20 teams competing in the challenge, and establishing a new strong baseline for future text-only AFC systems. Qualitative analysis of mislabeled instances reveals that InFact often yields a more accurate conclusion than AVeriTeC’s human-annotated ground truth.- Anthology ID:
- 2024.fever-1.12
- Volume:
- Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
- Venues:
- FEVER | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 108–112
- Language:
- URL:
- https://preview.aclanthology.org/fix-sig-urls/2024.fever-1.12/
- DOI:
- 10.18653/v1/2024.fever-1.12
- Cite (ACL):
- Mark Rothermel, Tobias Braun, Marcus Rohrbach, and Anna Rohrbach. 2024. InFact: A Strong Baseline for Automated Fact-Checking. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 108–112, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- InFact: A Strong Baseline for Automated Fact-Checking (Rothermel et al., FEVER 2024)
- PDF:
- https://preview.aclanthology.org/fix-sig-urls/2024.fever-1.12.pdf