@inproceedings{feng-etal-2026-mt3,
    title = "{MT}$^{3}$: A Synergistic Multi-Task {RL} Framework for Specializing {MLLM}s in Text Image Machine Translation",
    author = "Feng, Zhaopeng  and
      Liang, Yupu  and
      Cao, Shaosheng  and
      Su, Jiayuan  and
      Ren, Jiahan  and
      Zhou, Zhijie  and
      Huang, Wenxuan  and
      Wu, Jian  and
      Liu, Zuozhu",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.460/",
    pages = "10140--10157",
    ISBN = "979-8-89176-390-6",
    abstract = "Text Image Machine Translation (TIMT){---}the task of translating textual content embedded in images{---}is critical for applications in accessibility, cross-lingual information access, and real-world document understanding. However, TIMT remains a complex challenge due to the need for accurate optical character recognition (OCR), robust visual-text reasoning, and high-quality translation, often requiring cascading multi-stage pipelines. Recent advances in large-scale Reinforcement Learning (RL) have improved reasoning in Large Language Models (LLMs) and Multimodal LLMs (MLLMs), but their application to end-to-end TIMT is still underexplored. To bridge this gap, we introduce MT$^{3}$, a novel Multi-Task RL framework to specialize MLLMs into end-to-end expert TIMT models. MT$^{3}$ adopts a synergistic multi-task optimization paradigm targeting three key sub-skills: text recognition, context-aware reasoning, and translation. It is trained using a novel multi-mixed reward mechanism that provides fine-grained feedback, fostering a controllable and transparent optimization process. Furthermore, to facilitate the evaluation of TIMT in authentic cross-cultural and real-world social media contexts, we introduced XHSPost, the first social media TIMT benchmark. Our MT$^{3}$-7B-Zero achieves state-of-the-art results on the latest in-domain MIT-10M benchmark, outperforming strong baselines such as Qwen2.5-VL-72B and InternVL2.5-78B by notable margins across multiple metrics. Additionally, the model shows strong generalization to out-of-distribution language pairs and datasets. In-depth analyses reveal how multi-task synergy, reinforcement learning initialization, curriculum design, and reward formulation contribute to advancing MLLM-driven TIMT."
}