Multimodal Coreference Resolution for Chinese Social Media Dialogues: Dataset and Benchmark Approach

Xingyu Li, Chen Gong, Guohong Fu


Abstract
Multimodal coreference resolution (MCR) aims to identify mentions referring to the same entity across different modalities, such as text and visuals, and is essential for understanding multimodal content. In the era of rapidly growing multimodal content and social media, MCR is particularly crucial for interpreting user interactions and bridging text-visual references to improve communication and personalization. However, MCR research for real-world dialogues remains unexplored due to the lack of sufficient data resources. To address this gap, we introduce TikTalkCoref, the first Chinese multimodal coreference dataset for social media in real-world scenarios, derived from the popular Douyin short-video platform. This dataset pairs short videos with corresponding textual dialogues from user comments and includes manually annotated coreference clusters for both person mentions in the text and the coreferential person head regions in the corresponding video frames. We also present an effective benchmark approach for MCR, focusing on the celebrity domain, and conduct extensive experiments on our dataset, providing reliable benchmark results for this newly constructed dataset. We release the TikTalkCoref dataset to facilitate future research on MCR for real-world social media dialogues at https://github.com/lxystaruni/TikTalkCoref.
Anthology ID:
2025.acl-long.1520
Volume:
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Wanxiang Che, Joyce Nabende, Ekaterina Shutova, Mohammad Taher Pilehvar
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
31513–31525
Language:
URL:
https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1520/
DOI:
Bibkey:
Cite (ACL):
Xingyu Li, Chen Gong, and Guohong Fu. 2025. Multimodal Coreference Resolution for Chinese Social Media Dialogues: Dataset and Benchmark Approach. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 31513–31525, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
Multimodal Coreference Resolution for Chinese Social Media Dialogues: Dataset and Benchmark Approach (Li et al., ACL 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1520.pdf