@inproceedings{cheng-etal-2025-vision,
    title = "Vision-Language Models Can Self-Improve Reasoning via Reflection",
    author = "Cheng, Kanzhi  and
      YanTao, Li  and
      Xu, Fangzhi  and
      Zhang, Jianbing  and
      Zhou, Hao  and
      Liu, Yang",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.naacl-long.447/",
    pages = "8876--8892",
    ISBN = "979-8-89176-189-6",
    abstract = "Chain-of-thought (CoT) has proven to improve the reasoning capability of large language models (LLMs). However, due to the complexity of multimodal scenarios and the difficulty in collecting high-quality CoT data, CoT reasoning in multimodal LLMs has been largely overlooked. To this end, we propose a simple yet effective self-training framework, $R^3V$, which iteratively enhances the model{'}s Vision-language Reasoning by Reflecting on CoT Rationales. Our framework consists of two interleaved parts: (1) iteratively bootstrapping positive and negative solutions for reasoning datasets, and (2) reflection on rationale for learning from mistakes. Specifically, we introduce the self-refine and self-select losses, enabling the model to refine flawed rationale and derive the correct answer by comparing rationale candidates. Experiments on a wide range of vision-language tasks show that $R^3V$ consistently improves multimodal LLM reasoning, achieving a relative improvement of 23{\%} to 60{\%} over GPT-distilled baselines. Additionally, our approach supports self-reflection on generated solutions, further boosting performance through test-time computation. Our code is available at https://github.com/njucckevin/MM-Self-Improve."
}