Exploring Large Language Models for Effective Rumor Detection on Social Media

Yirong Zeng, Xiao Ding, Bibo Cai, Ting Liu, Bing Qin


Abstract
In this paper, we explore using Large Language Models (LLMs) for rumor detection on social media. It involves assessing the veracity of claims on social media based on social context (e.g., comments, propagation patterns). LLMs, despite their impressive capabilities in text-based reasoning tasks, struggle to achieve promising rumor detection performance when facing long structured social contexts. Our preliminary analysis shows that large-scale contexts hinder LLMs’ reasoning abilities, while moderate contexts perform better for LLMs, highlighting the need for refined contexts. Accordingly, we propose a semantic-propagation collaboration-base framework that integrates small language models (e.g., graph attention network) with LLMs for effective rumor detection. It models contexts by enabling text semantic and propagation patterns to collaborate through graph attention mechanisms, and reconstruct the context by aggregating attention values during inference. Also, a cluster-based unsupervised method to refine context is proposed for generalization. Extensive experiments demonstrate the effectiveness of proposed methods in rumor detection. This work bridges the gap for LLMs in facing long, structured data and offers a novel solution for rumor detection on social media.
Anthology ID:
2025.naacl-long.128
Volume:
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Month:
April
Year:
2025
Address:
Albuquerque, New Mexico
Editors:
Luis Chiruzzo, Alan Ritter, Lu Wang
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2537–2552
Language:
URL:
https://preview.aclanthology.org/fix-sig-urls/2025.naacl-long.128/
DOI:
Bibkey:
Cite (ACL):
Yirong Zeng, Xiao Ding, Bibo Cai, Ting Liu, and Bing Qin. 2025. Exploring Large Language Models for Effective Rumor Detection on Social Media. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2537–2552, Albuquerque, New Mexico. Association for Computational Linguistics.
Cite (Informal):
Exploring Large Language Models for Effective Rumor Detection on Social Media (Zeng et al., NAACL 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/fix-sig-urls/2025.naacl-long.128.pdf