@inproceedings{sinha-etal-2026-minds,
    title = "Mind{'}s Eye: A Benchmark of Visual Abstraction, Transformation and Composition for Multimodal {LLM}s",
    author = "Sinha, Rohit  and
      Kanade, Aditya Sanjiv  and
      Kancheti, Sai Srinivas  and
      Balasubramanian, Vineeth N.  and
      Ganu, Tanuja",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.2124/",
    pages = "45794--45835",
    ISBN = "979-8-89176-390-6",
    abstract = "Multimodal large language models (MLLMs) have achieved impressive progress on vision language benchmarks, yet their capacity for \textit{cognitive} and \textit{psychological} reasoning remains largely unexplored. We introduce Mind{'}s Eye, a multiple-choice benchmark of eight visuo-cognitive tasks inspired by classic human intelligence tests and organized under a novel \textbf{A{--}R{--}T} taxonomy: \textbf{Abstraction}, \textbf{Relation}, and \textbf{Transformation}. The tasks probe core processes of fluid intelligence such as pattern induction, analogical Relation mapping, and mental Transformation. We evaluate a diverse suite of closed-source and open-source MLLMs and compare their performance with human participants. Humans achieve \textbf{80{\%}} accuracy, while top performing MLLMs remain below \textbf{50{\%}}. Error analysis reveals failures in (i) visual attention allocation, (ii) internal perceptual manipulation, (iii) over reliance on domain priors, and (iv) weak abstraction of underlying visual concepts. Our findings suggest that current MLLMs exhibit limited fluid reasoning and visuo-cognitive integration compared with human participants, highlighting the need for cognitively grounded evaluation frameworks like Mind{'}s Eye."
}