@inproceedings{du-etal-2026-multimodal,
    title = "Multimodal Generative Engine Optimization: Rank Manipulation for Vision{--}Language Model Rankers",
    author = "Du, Yixuan  and
      Yu, Chenxiao  and
      Xu, Haoyan  and
      Wang, Ziyi  and
      Zhao, Yue  and
      Hu, Xiyang",
    editor = "Chen, Canyu  and
      Zhang, Yuji  and
      Li, Zoey Sha  and
      Wang, Zihan  and
      Wang, Qineng  and
      Su, Jinyan  and
      Kargupta, Priyanka  and
      Marjanovi{\'c}, Sara Vera  and
      Pan, Jeff Z.  and
      Bansal, Mohit  and
      Augenstein, Isabelle  and
      Han, Jiawei  and
      Ji, Heng  and
      Li, Manling",
    booktitle = "Proceedings of the 4th Workshop on Towards Knowledgeable Foundation Models ({K}now{FM} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.knowfm-1.9/",
    pages = "115--128",
    ISBN = "979-8-89176-403-3",
    abstract = "Vision-Language Models (VLMs) integrate visual and textual knowledge into unified representations that increasingly underpin modern retrieval and recommendation systems. However, it remains unclear how reliably these models utilize their cross-modal knowledge when ranking multimodal items, and whether their knowledge grounding can be subverted. In this paper, we expose a fundamental vulnerability in how VLMs apply multimodal knowledge for product ranking: through Multimodal Generative Engine Optimization (MGEO), we show that an adversary can manipulate a VLM{'}s ranking decisions by jointly crafting imperceptible image perturbations and fluent textual suffixes that exploit the model{'}s internal cross-modal knowledge coupling. Using an alternating optimization strategy, MGEO targets the deep interactions between visual and linguistic representations within the VLM, achieving rank manipulations that substantially exceed those of unimodal attacks and heuristic baselines powered by strong commercial models. Our findings reveal that surface-level content quality is insufficient for rank promotion; instead, direct alignment with the model{'}s internal knowledge utilization mechanism is required. These results raise important questions on the faithfulness and robustness of knowledge grounding in multimodal foundation models, and motivate future work on defense mechanisms for multimodal retrieval systems."
}