@inproceedings{zhao-etal-2025-omnialign,
    title = "{O}mni{A}lign-{V}: Towards Enhanced Alignment of {MLLM}s with Human Preference",
    author = "Zhao, Xiangyu  and
      Ding, Shengyuan  and
      Zhang, Zicheng  and
      Huang, Haian  and
      Maosongcao, Maosongcao  and
      Wang, Jiaqi  and
      Wang, Weiyun  and
      Fang, Xinyu  and
      Wang, Wenhai  and
      Zhai, Guangtao  and
      Yang, Hua  and
      Duan, Haodong  and
      Chen, Kai",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.906/",
    pages = "18490--18515",
    ISBN = "979-8-89176-251-0",
    abstract = "Recent advancements in open-source multi-modal large language models (MLLMs) have primarily focused on enhancing foundational capabilities, leaving a significant gap in human preference alignment. This paper introduces OmniAlign-V, a comprehensive dataset of 200K high-quality training samples featuring diverse images, complex questions, and varied response formats to improve MLLMs' alignment with human preferences. We also present MM-AlignBench, a human-annotated benchmark specifically designed to evaluate MLLMs' alignment with human values. Experimental results show that finetuning MLLMs with OmniAlign-V, using Supervised Fine-Tuning (SFT) or Direct Preference Optimization (DPO), significantly enhances human preference alignment while maintaining or enhancing performance on standard VQA benchmarks, preserving their fundamental capabilities."
}