@inproceedings{choi-etal-2026-users,
    title = "What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models",
    author = "Choi, Dasol  and
      Son, Guijin  and
      Lee, Hanwool  and
      Kim, Minhyuk  and
      Ko, Hyunwoo  and
      Lim, Teabin  and
      Ahn, Eungyeol  and
      Kim, Jungwhan  and
      Hong, Seunghyeok  and
      Song, Youngsook",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.528/",
    pages = "10863--10886",
    ISBN = "979-8-89176-395-1",
    abstract = "Current vision-language benchmarks predominantly feature well-structured questions with clear, explicit prompts. However, real user queries are often informal and underspecified. Users naturally leave much unsaid, relying on images to convey context. We introduce HAERAE-Vision, a benchmark of 653 real-world visual questions from Korean online communities (0.76{\%} survival from 86K candidates), each paired with an explicit rewrite, yielding 1,306 query variants in total. Evaluating 39 VLMs, we find that even state-of-the-art models (GPT-5, Gemini 2.5 Pro) achieve under 50{\%} on the original queries. Crucially, query explicitation alone yields 8 to 22 point improvements, with smaller models benefiting most. We further show that even with web search, under-specified queries underperform explicit queries without search, revealing that current retrieval cannot compensate for what users leave unsaid. Our findings demonstrate that a substantial portion of VLM difficulty stem from natural query under-specification instead of model capability, highlighting a critical gap between benchmark evaluation and real-world deployment."
}