Abstract
To tackle the AVeriTeC shared task hosted by the FEVER-24, we introduce a system that only employs publicly available large language models (LLMs) for each step of automated fact-checking, dubbed the Herd of Open LLMs for verifying real-world claims (HerO). HerO employs multiple LLMs for each step of automated fact-checking. For evidence retrieval, a language model is used to enhance a query by generating hypothetical documents that check the veracity of a claim. We fine-tune LLMs for question generation and veracity prediction by crafting prompts with retrieved in-context samples. HerO achieved 2nd place on the leaderboard with the AVeriTeC score of 0.57, suggesting the potential of open LLMs for verifying real-world claims. For future research, we make our code publicly available at https://github.com/ssu-humane/HerO.- Anthology ID:
- 2024.fever-1.15
- Volume:
- Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
- Venue:
- FEVER
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 130–136
- Language:
- URL:
- https://aclanthology.org/2024.fever-1.15
- DOI:
- 10.18653/v1/2024.fever-1.15
- Cite (ACL):
- Yejun Yoon, Jaeyoon Jung, Seunghyun Yoon, and Kunwoo Park. 2024. HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claims. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 130–136, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claims (Yoon et al., FEVER 2024)
- PDF:
- https://preview.aclanthology.org/landing_page/2024.fever-1.15.pdf