FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German
Justus Mattern, Yu Qiao, Elma Kerz, Daniel Wiechmann, Markus Strohmaier
Abstract
As the world continues to fight the COVID-19 pandemic, it is simultaneously fighting an ‘infodemic’ – a flood of disinformation and spread of conspiracy theories leading to health threats and the division of society. To combat this infodemic, there is an urgent need for benchmark datasets that can help researchers develop and evaluate models geared towards automatic detection of disinformation. While there are increasing efforts to create adequate, open-source benchmark datasets for English, comparable resources are virtually unavailable for German, leaving research for the German language lagging significantly behind. In this paper, we introduce the new benchmark dataset FANG-COVID consisting of 28,056 real and 13,186 fake German news articles related to the COVID-19 pandemic as well as data on their propagation on Twitter. Furthermore, we propose an explainable textual- and social context-based model for fake news detection, compare its performance to “black-box” models and perform feature ablation to assess the relative importance of human-interpretable features in distinguishing fake news from authentic news.- Anthology ID:
- 2021.fever-1.9
- Volume:
- Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)
- Month:
- November
- Year:
- 2021
- Address:
- Dominican Republic
- Editors:
- Rami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlichtkrull, James Thorne, Andreas Vlachos
- Venue:
- FEVER
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 78–91
- Language:
- URL:
- https://aclanthology.org/2021.fever-1.9
- DOI:
- 10.18653/v1/2021.fever-1.9
- Cite (ACL):
- Justus Mattern, Yu Qiao, Elma Kerz, Daniel Wiechmann, and Markus Strohmaier. 2021. FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German. In Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER), pages 78–91, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German (Mattern et al., FEVER 2021)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2021.fever-1.9.pdf
- Code
- justusmattern/fang-covid
- Data
- CoAID