CLFFRD: Curriculum Learning and Fine-grained Fusion for Multimodal Rumor Detection

Fan Xu, Lei Zeng, Bowei Zou, Ai Ti Aw, Huan Rong

Abstract
In an era where rumors can propagate rapidly across social media platforms such as Twitter and Weibo, automatic rumor detection has garnered considerable attention from both academia and industry. Existing multimodal rumor detection models often overlook the intricacies of sample difficulty, e.g., text-level difficulty, image-level difficulty, and multimodal-level difficulty, as well as their order when training. Inspired by the concept of curriculum learning, we propose the Curriculum Learning and Fine-grained Fusion-driven multimodal Rumor Detection (CLFFRD) framework, which employs curriculum learning to automatically select and train samples according to their difficulty at different training stages. Furthermore, we introduce a fine-grained fusion strategy that unifies entities from text and objects from images, enhancing their semantic cohesion. We also propose a novel data augmentation method that utilizes linear interpolation between textual and visual modalities to generate diverse data. Additionally, our approach incorporates deep fusion for both intra-modality (e.g., text entities and image objects) and inter-modality (e.g., CLIP and social graph) features. Extensive experimental results demonstrate that CLFFRD outperforms state-of-the-art models on both English and Chinese benchmark datasets for rumor detection in social media.
Anthology ID:
2024.lrec-main.294
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
3314–3324
Language:
URL:
https://aclanthology.org/2024.lrec-main.294
DOI:
Bibkey:
Cite (ACL):
Fan Xu, Lei Zeng, Bowei Zou, Ai Ti Aw, and Huan Rong. 2024. CLFFRD: Curriculum Learning and Fine-grained Fusion for Multimodal Rumor Detection. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 3314–3324, Torino, Italia. ELRA and ICCL.
Cite (Informal):
CLFFRD: Curriculum Learning and Fine-grained Fusion for Multimodal Rumor Detection (Xu et al., LREC-COLING 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.lrec-main.294.pdf
PDF Search