@inproceedings{otake-etal-2025-bannerbench,
    title = "{B}anner{B}ench: Benchmarking Vision Language Models for Multi-Ad Selection with Human Preferences",
    author = "Otake, Hiroto  and
      Zhang, Peinan  and
      Sakai, Yusuke  and
      Mita, Masato  and
      Ouchi, Hiroki  and
      Watanabe, Taro",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1311/",
    doi = "10.18653/v1/2025.findings-emnlp.1311",
    pages = "24145--24159",
    ISBN = "979-8-89176-335-7",
    abstract = "Web banner advertisements, which are placed on websites to guide users to a targeted landing page (LP), are still often selected manually because human preferences are important in selecting which ads to deliver. To automate this process, we propose a new benchmark, BannerBench, to evaluate the human preference-driven banner selection process using vision-language models (VLMs). This benchmark assesses the degree of alignment with human preferences in two tasks: a ranking task and a best-choice task, both using sets of five images derived from a single LP. Our experiments show that VLMs are moderately correlated with human preferences on the ranking task. In the best-choice task, most VLMs perform close to chance level across various prompting strategies. These findings suggest that although VLMs have a basic understanding of human preferences, most of them struggle to pinpoint a single suitable option from many candidates."
}