DART: Disambiguation-Aware Reasoning for Video-guided Machine Translation

Boyu Guan, Chuang Han, Yang Zhao, Chengqing Zong


Abstract
Video-guided Machine Translation (VMT) seeks to enhance translation quality by incorporating contextual information derived from paired short video clips. However, many VMT samples are text-sufficient; even when visual information is needed, only minimal cues are required. Aiming to tackle these issues, we propose a novel framework **DART** (**D**isambiguation-**A**ware **R**easoning for Video-guided Machine **T**ranslation). Reinforcement learning is used to incorporate multimodal large language models’ multimodal reasoning into VMT. The model dynamically switches between text-only processing and multimodal integration, contingent on the necessity of visual disambiguation. Furthermore, we present **TVRF** (**T**ranslation-oriented **V**ideo **R**elevance **F**iltering), a systematic pipeline for constructing training data based on multimodal relevance to translation. This pipeline filters samples where video information is translation-relevant, mitigating training collapse caused by video-irrelevant data in conventional VMT. Experimental results show that our approach improves multimodal information utilization in VMT, yielding gains in both translation quality and computational efficiency.
Anthology ID:
2026.acl-long.352
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7752–7772
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.352/
DOI:
Bibkey:
Cite (ACL):
Boyu Guan, Chuang Han, Yang Zhao, and Chengqing Zong. 2026. DART: Disambiguation-Aware Reasoning for Video-guided Machine Translation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7752–7772, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
DART: Disambiguation-Aware Reasoning for Video-guided Machine Translation (Guan et al., ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.352.pdf
Checklist:
 2026.acl-long.352.checklist.pdf