MM-AVS: A Full-Scale Dataset for Multi-modal Summarization

Xiyan Fu, Jun Wang, Zhenglu Yang


Abstract
Multimodal summarization becomes increasingly significant as it is the basis for question answering, Web search, and many other downstream tasks. However, its learning materials have been lacking a holistic organization by integrating resources from various modalities, thereby lagging behind the research progress of this field. In this study, we release a full-scale multimodal dataset comprehensively gathering documents, summaries, images, captions, videos, audios, transcripts, and titles in English from CNN and Daily Mail. To our best knowledge, this is the first collection that spans all modalities and nearly comprises all types of materials available in this community. In addition, we devise a baseline model based on the novel dataset, which employs a newly proposed Jump-Attention mechanism based on transcripts. The experimental results validate the important assistance role of the external information for multimodal summarization.
Anthology ID:
2021.naacl-main.473
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5922–5926
Language:
URL:
https://aclanthology.org/2021.naacl-main.473
DOI:
10.18653/v1/2021.naacl-main.473
Bibkey:
Cite (ACL):
Xiyan Fu, Jun Wang, and Zhenglu Yang. 2021. MM-AVS: A Full-Scale Dataset for Multi-modal Summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5922–5926, Online. Association for Computational Linguistics.
Cite (Informal):
MM-AVS: A Full-Scale Dataset for Multi-modal Summarization (Fu et al., NAACL 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.naacl-main.473.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2021.naacl-main.473.mp4
Data
CNN/Daily MailHow2