VISA: An Ambiguous Subtitles Dataset for Visual Scene-aware Machine Translation
Yihang Li, Shuichiro Shimizu, Weiqi Gu, Chenhui Chu, Sadao Kurohashi
Abstract
Existing multimodal machine translation (MMT) datasets consist of images and video captions or general subtitles which rarely contain linguistic ambiguity, making visual information not so effective to generate appropriate translations. We introduce VISA, a new dataset that consists of 40k Japanese-English parallel sentence pairs and corresponding video clips with the following key features: (1) the parallel sentences are subtitles from movies and TV episodes; (2) the source subtitles are ambiguous, which means they have multiple possible translations with different meanings; (3) we divide the dataset into Polysemy and Omission according to the cause of ambiguity. We show that VISA is challenging for the latest MMT system, and we hope that the dataset can facilitate MMT research.- Anthology ID:
- 2022.lrec-1.725
- Volume:
- Proceedings of the Thirteenth Language Resources and Evaluation Conference
- Month:
- June
- Year:
- 2022
- Address:
- Marseille, France
- Editors:
- Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 6735–6743
- Language:
- URL:
- https://aclanthology.org/2022.lrec-1.725
- DOI:
- Cite (ACL):
- Yihang Li, Shuichiro Shimizu, Weiqi Gu, Chenhui Chu, and Sadao Kurohashi. 2022. VISA: An Ambiguous Subtitles Dataset for Visual Scene-aware Machine Translation. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6735–6743, Marseille, France. European Language Resources Association.
- Cite (Informal):
- VISA: An Ambiguous Subtitles Dataset for Visual Scene-aware Machine Translation (Li et al., LREC 2022)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2022.lrec-1.725.pdf
- Code
- ku-nlp/visa
- Data
- How2, OpenSubtitles