@inproceedings{gong-etal-2026-probing,
    title = "Probing Audio-Visual Reasoning in Multimodal Language Models through the Lens of Audio",
    author = "Gong, Kaixiong  and
      Feng, Kaituo  and
      Li, Bohao  and
      Wang, Yibing  and
      Cheng, Mofan  and
      Yang, Shijia  and
      Han, Jiaming  and
      Wang, Benyou  and
      Bai, Yutong  and
      Yang, Zhuoran  and
      Yue, Xiangyu",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1697/",
    pages = "36603--36645",
    ISBN = "979-8-89176-390-6",
    abstract = "Recent multimodal large language models (MLLMs), such as GPT-4o, Gemini 1.5/2.5 Pro, and Reka Core, have advanced audio-visual reasoning capabilities, achieving strong performance in tasks like cross-modal understanding and generation. However, our \textbf{DeafTest} uncovers unanticipated failures: most of the state-of-the-art MLLMs struggle with very simple audio tasks, such as \textit{distinguishing louder sounds} or \textit{sound counting}. This raises a fundamental question{---}does a deficiency in low-level audio perception constrain higher-level audio-visual reasoning? To address this, we introduce \textbf{AV-Odyssey Bench}{---}a comprehensive benchmark of 4,555 meticulously designed problems that integrate text, audio, and visual modalities. Each task requires models to unify cross-modal reasoning, leveraging synchronized audio-visual cues to infer solutions. By structuring questions as multiple-choice, we ensure objective, reproducible evaluations without reliance on subjective human or LLM-based judgments. Through comprehensive benchmarking of closed-source and open-source models, we showcase: (i) current MLLMs lack robust audio-visual integration ability and (ii) performance on DeafTest (Pearson{'}s $r = 0.945$) strongly correlates with AV-Odyssey accuracy. These findings challenge assumptions about models' multimodal proficiency and highlight fundamental audio perception as a reasoning bottleneck. We believe that our results provide concrete guidance for future dataset design, alignment strategies, and architectures."
}