@inproceedings{zhou-etal-2025-chatvla,
    title = "{C}hat{VLA}: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model",
    author = "Zhou, Zhongyi  and
      Zhu, Yichen  and
      Zhu, Minjie  and
      Wen, Junjie  and
      Liu, Ning  and
      Xu, Zhiyuan  and
      Meng, Weibin  and
      Peng, Yaxin  and
      Shen, Chaomin  and
      Feng, Feifei  and
      Xu, Yi",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.273/",
    pages = "5377--5395",
    ISBN = "979-8-89176-332-6",
    abstract = "Humans possess a unified cognitive ability to perceive, comprehend, and interact with the physical world. Why can{'}t large language models replicate this holistic understanding? Through a systematic analysis of existing training paradigms in vision-language-action models (VLA), we identify two key challenges: \textit{spurious forgetting}, where robot training overwrites crucial visual-text alignments, and \textit{task interference}, where competing control and understanding tasks degrade performance when trained jointly. To overcome these limitations, we propose ChatVLA, a novel framework featuring Phased Alignment Training, which incrementally integrates multimodal data after initial control mastery, and a Mixture-of-Experts architecture to minimize task interference. ChatVLA demonstrates competitive performance on visual question-answering datasets and significantly surpasses state-of-the-art vision-language-action (VLA) methods on multimodal understanding benchmarks. Notably, it achieves a six times higher performance on MMMU and scores 47.2{\%} on MMStar with a more parameter-efficient design than ECoT. Furthermore, ChatVLA demonstrates superior performance on 25 real-world robot manipulation tasks compared to existing VLA methods like OpenVLA. Our findings highlight the potential of our unified framework for achieving both robust multimodal understanding and effective robot control."
}