Abstract
This study investigates how fake news use the thumbnail image for a news article. We aim at capturing the degree of semantic incongruity between news text and image by using the pretrained CLIP representation. Motivated by the stylistic distinctiveness in fake news text, we examine whether fake news tends to use an irrelevant image to the news content. Results show that fake news tends to have a high degree of semantic incongruity than general news. We further attempt to detect such image-text incongruity by training classification models on a newly generated dataset. A manual evaluation suggests our method can find news articles of which the thumbnail image is semantically irrelevant to news text with an accuracy of 0.8. We also release a new dataset of image and news text pairs with the incongruity label, facilitating future studies on the direction.- Anthology ID:
- 2022.constraint-1.10
- Volume:
- Proceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Tanmoy Chakraborty, Md. Shad Akhtar, Kai Shu, H. Russell Bernard, Maria Liakata, Preslav Nakov, Aseem Srivastava
- Venue:
- CONSTRAINT
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 86–94
- Language:
- URL:
- https://aclanthology.org/2022.constraint-1.10
- DOI:
- 10.18653/v1/2022.constraint-1.10
- Cite (ACL):
- Hyewon Choi, Yejun Yoon, Seunghyun Yoon, and Kunwoo Park. 2022. How does fake news use a thumbnail? CLIP-based Multimodal Detection on the Unrepresentative News Image. In Proceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations, pages 86–94, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- How does fake news use a thumbnail? CLIP-based Multimodal Detection on the Unrepresentative News Image (Choi et al., CONSTRAINT 2022)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2022.constraint-1.10.pdf
- Code
- ssu-humane/fake-news-thumbnail