@inproceedings{villa-cueva-etal-2025-moments,
    title = "{M}o{M}ent{S}: A Comprehensive Multimodal Benchmark for Theory of Mind",
    author = "Villa-Cueva, Emilio  and
      Ahmed, S M Masrur  and
      Chevi, Rendi  and
      Cruz, Jan Christian Blaise  and
      Elzeky, Kareem  and
      Cristobal, Fermin  and
      Aji, Alham Fikri  and
      Wang, Skyler  and
      Mihalcea, Rada  and
      Solorio, Thamar",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1230/",
    doi = "10.18653/v1/2025.findings-emnlp.1230",
    pages = "22591--22611",
    ISBN = "979-8-89176-335-7",
    abstract = "Understanding Theory of Mind is essential for building socially intelligent multimodal agents capable of perceiving and interpreting human behavior. We introduce MoMentS (Multimodal Mental States), a comprehensive benchmark designed to assess the ToM capabilities of multimodal large language models (LLMs) through realistic, narrative-rich scenarios presented in short films. MoMentS includes over 2,300 multiple-choice questions spanning seven distinct ToM categories. The benchmark features long video context windows and realistic social interactions that provide deeper insight into characters' mental states. We evaluate several MLLMs and find that although vision generally improves performance, models still struggle to integrate it effectively. For audio, models that process dialogues as audio do not consistently outperform transcript-based inputs. Our findings highlight the need to improve multimodal integration and point to open challenges that must be addressed to advance AI{'}s social understanding."
}