SINCon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor Detection

Mingqing Zhang, Qiang Liu, Xiang Tao, Shu Wu, Liang Wang


Abstract
In the era of rapidly evolving large language models (LLMs), state-of-the-art rumor detection systems, particularly those based on Message Propagation Trees (MPTs), which represent a conversation tree with the post as its root and the replies as its descendants, are facing increasing threats from adversarial attacks that leverage LLMs to generate and inject malicious messages. Existing methods are based on the assumption that different nodes exhibit varying degrees of influence on predictions. They define nodes with high predictive influence as important nodes and target them for attacks. If the model treats nodes’ predictive influence more uniformly, attackers will find it harder to target high predictive influence nodes. In this paper, we propose Similarizing the predictive Influence of Nodes with Contrastive Learning (SINCon), a defense mechanism that encourages the model to learn graph representations where nodes with varying importance have a more uniform influence on predictions. Extensive experiments on the Twitter and Weibo datasets demonstrate that SINCon not only preserves high classification accuracy on clean data but also significantly enhances resistance against LLM-driven message injection attacks.
Anthology ID:
2025.acl-long.617
Volume:
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Wanxiang Che, Joyce Nabende, Ekaterina Shutova, Mohammad Taher Pilehvar
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
12570–12581
Language:
URL:
https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.617/
DOI:
Bibkey:
Cite (ACL):
Mingqing Zhang, Qiang Liu, Xiang Tao, Shu Wu, and Liang Wang. 2025. SINCon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor Detection. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12570–12581, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
SINCon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor Detection (Zhang et al., ACL 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.617.pdf