Language Models as Fact Checkers?
Nayeon Lee, Belinda Z. Li, Sinong Wang, Wen-tau Yih, Hao Ma, Madian Khabsa
Abstract
Recent work has suggested that language models (LMs) store both common-sense and factual knowledge learned from pre-training data. In this paper, we leverage this implicit knowledge to create an effective end-to-end fact checker using a solely a language model, without any external knowledge or explicit retrieval components. While previous work on extracting knowledge from LMs have focused on the task of open-domain question answering, to the best of our knowledge, this is the first work to examine the use of language models as fact checkers. In a closed-book setting, we show that our zero-shot LM approach outperforms a random baseline on the standard FEVER task, and that our finetuned LM compares favorably with standard baselines. Though we do not ultimately outperform methods which use explicit knowledge bases, we believe our exploration shows that this method is viable and has much room for exploration.- Anthology ID:
- 2020.fever-1.5
- Volume:
- Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER)
- Month:
- July
- Year:
- 2020
- Address:
- Online
- Editors:
- Christos Christodoulopoulos, James Thorne, Andreas Vlachos, Oana Cocarascu, Arpit Mittal
- Venue:
- FEVER
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 36–41
- Language:
- URL:
- https://aclanthology.org/2020.fever-1.5
- DOI:
- 10.18653/v1/2020.fever-1.5
- Cite (ACL):
- Nayeon Lee, Belinda Z. Li, Sinong Wang, Wen-tau Yih, Hao Ma, and Madian Khabsa. 2020. Language Models as Fact Checkers?. In Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER), pages 36–41, Online. Association for Computational Linguistics.
- Cite (Informal):
- Language Models as Fact Checkers? (Lee et al., FEVER 2020)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2020.fever-1.5.pdf
- Data
- FEVER, LAMA