The 2nd Automated Verification of Textual Claims (AVeriTeC) Shared Task: Open-weights, Reproducible and Efficient Systems
Mubashara Akhtar, Rami Aly, Yulong Chen, Zhenyun Deng, Michael Schlichtkrull, Chenxi Whitehouse, Andreas Vlachos
Abstract
In the First Automated Verification of Textual Claims (AVeriTeC) shared task participanting teams developed systems that for each claim retrieve evidence from the web and predict its veracity. While there was progress in automated fact-checking for real-world claims, the majority of the systems proposed relied on closed-weights large language models, which rendered them expensive to run and less reporducible. To ameliorate this issue, in this year’s edition of the AVERITEC shared task we required system to use only open-weights models that could be run use a single GPU with 23GBs of RAM, and that systems should take one minute or less to return verdicts accompanied by evidence retrieved from a precompiled knowledge store. The shared task received 7 submissions; 6 of which exceeded the accuracy of our baseline on the test set, while they ran in under a minute per claim on the hardware we had speficied. The winning team was CTU AIC with an AVeriTeC score of 33.17%. In this paper we describe the shared task in detail and highlight key findings.- Anthology ID:
- 2025.fever-1.15
- Volume:
- Proceedings of the Eighth Fact Extraction and VERification Workshop (FEVER)
- Month:
- July
- Year:
- 2025
- Address:
- Vienna, Austria
- Editors:
- Mubashara Akhtar, Rami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlichtkrull, James Thorne, Andreas Vlachos
- Venues:
- FEVER | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 201–223
- Language:
- URL:
- https://preview.aclanthology.org/acl25-workshop-ingestion/2025.fever-1.15/
- DOI:
- Cite (ACL):
- Mubashara Akhtar, Rami Aly, Yulong Chen, Zhenyun Deng, Michael Schlichtkrull, Chenxi Whitehouse, and Andreas Vlachos. 2025. The 2nd Automated Verification of Textual Claims (AVeriTeC) Shared Task: Open-weights, Reproducible and Efficient Systems. In Proceedings of the Eighth Fact Extraction and VERification Workshop (FEVER), pages 201–223, Vienna, Austria. Association for Computational Linguistics.
- Cite (Informal):
- The 2nd Automated Verification of Textual Claims (AVeriTeC) Shared Task: Open-weights, Reproducible and Efficient Systems (Akhtar et al., FEVER 2025)
- PDF:
- https://preview.aclanthology.org/acl25-workshop-ingestion/2025.fever-1.15.pdf