Abstract
Fact-checking is a journalistic practice that compares a claim made publicly against trusted sources of facts. Wang (2017) introduced a large dataset of validated claims from the POLITIFACT.com website (LIAR dataset), enabling the development of machine learning approaches for fact-checking. However, approaches based on this dataset have focused primarily on modeling the claim and speaker-related metadata, without considering the evidence used by humans in labeling the claims. We extend the LIAR dataset by automatically extracting the justification from the fact-checking article used by humans to label a given claim. We show that modeling the extracted justification in conjunction with the claim (and metadata) provides a significant improvement regardless of the machine learning model used (feature-based or deep learning) both in a binary classification task (true, false) and in a six-way classification task (pants on fire, false, mostly false, half true, mostly true, true).- Anthology ID:
- W18-5513
- Volume:
- Proceedings of the First Workshop on Fact Extraction and VERification (FEVER)
- Month:
- November
- Year:
- 2018
- Address:
- Brussels, Belgium
- Editors:
- James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, Arpit Mittal
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 85–90
- Language:
- URL:
- https://aclanthology.org/W18-5513
- DOI:
- 10.18653/v1/W18-5513
- Cite (ACL):
- Tariq Alhindi, Savvas Petridis, and Smaranda Muresan. 2018. Where is Your Evidence: Improving Fact-checking by Justification Modeling. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pages 85–90, Brussels, Belgium. Association for Computational Linguistics.
- Cite (Informal):
- Where is Your Evidence: Improving Fact-checking by Justification Modeling (Alhindi et al., EMNLP 2018)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/W18-5513.pdf
- Code
- Tariq60/LIAR-PLUS + additional community code
- Data
- LIAR