@inproceedings{rodriguez-salgado-2026-pixels,
    title = "From Pixels to {BFS}: High Maze Accuracy Does Not Imply Visual Planning",
    author = "Rodriguez Salgado, Alberto Gonzalo",
    editor = "Yan, Qianqi  and
      Montariol, Syrielle  and
      Fan, Yue  and
      Gu, Jing  and
      Pan, Jiayi  and
      Li, Manling  and
      Kordjamshidi, Parisa  and
      Suhr, Alane  and
      Wang, Xin Eric",
    booktitle = "Proceedings of the 4th Workshop on Advances in Language and Vision Research ({ALVR})",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.alvr-main.13/",
    pages = "164--179",
    ISBN = "979-8-89176-398-2",
    abstract = "How do multimodal models solve visual spatial tasks{---}through genuine planning, or by brute-forcing solutions in token space? We introduce MazeBench, a benchmark of 110 procedurally generated maze images organized into nine controlled groups (diagnostic, grid scale, wall density, trap ablation, unreachable detection, and more), and evaluate 16 model configurations across four providers (OpenAI, Anthropic, Google, Alibaba) at multiple reasoning effort levels. GPT-5.4 solves 91{\%} and Gemini 3.1 Pro 79{\%}, but our analysis reveals these scores are misleading: models translate images into text grids and brute-force paths via serial enumeration, consuming 1,710{--}22,818 tokens per solve for a task humans do in seconds. Without added reasoning budgets, all configurations score only 2{--}12{\%}; on 20x20 ultra-hard mazes, they hit token limits and give up. Qualitative analysis of model outputs confirms a universal two-stage strategy: image-to-grid translation followed by step-by-step path search in natural language{---}essentially BFS implemented in prose. A text-grid ablation shows Claude{'}s poor image performance (6{\%}) jumps to 80{\%} when given the correct grid directly, confirming vision quality, not reasoning ability, as the bottleneck for weaker models. Perhaps most striking, when we explicitly instruct models not to build a text grid and not to perform graph search{---}asking them to ``reason visually, like a human''{---}they silently ignore the instruction and immediately fall back to the same grid-enumeration strategy. This suggests that brute-force token-level search is the dominant mechanism these models rely on for spatial planning in our setting."
}