SK_DU Team: Cross-Encoder based Evidence Retrieval and Question Generation with Improved Prompt for the AVeriTeC Shared Task

Shrikant Malviya, Stamos Katsigiannis

Abstract
As part of the AVeriTeC shared task, we developed a pipelined system comprising robust and finely tuned models. Our system integrates advanced techniques for evidence retrieval and question generation, leveraging cross-encoders and large language models (LLMs) for optimal performance. With multi-stage processing, the pipeline demonstrates improvements over baseline models, particularly in handling complex claims that require nuanced reasoning by improved evidence extraction, question generation and veracity prediction. Through detailed experiments and ablation studies, we provide insights into the strengths and weaknesses of our approach, highlighting the critical role of evidence sufficiency and context dependency in automated fact-checking systems. Our system secured a competitive rank, 7th on the development and 12th on the test data, in the shared task, underscoring the effectiveness of our methods in addressing the challenges of real-world claim verification.
Anthology ID:
2024.fever-1.11
Volume:
Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER)
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Michael Schlichtkrull, Yulong Chen, Chenxi Whitehouse, Zhenyun Deng, Mubashara Akhtar, Rami Aly, Zhijiang Guo, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
99–107
Language:
URL:
https://preview.aclanthology.org/add_missing_videos/2024.fever-1.11/
DOI:
10.18653/v1/2024.fever-1.11
Bibkey:
Cite (ACL):
Shrikant Malviya and Stamos Katsigiannis. 2024. SK_DU Team: Cross-Encoder based Evidence Retrieval and Question Generation with Improved Prompt for the AVeriTeC Shared Task. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), pages 99–107, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
SK_DU Team: Cross-Encoder based Evidence Retrieval and Question Generation with Improved Prompt for the AVeriTeC Shared Task (Malviya & Katsigiannis, FEVER 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/add_missing_videos/2024.fever-1.11.pdf
PDF Search Fix data