@inproceedings{tam-etal-2026-vistw,
    title = "{V}is{TW}: Benchmarking Vision-Language Models for {T}aiwanese {M}andarin in {T}aiwan",
    author = "Tam, Zhi Rui  and
      Shih, Yung-Yu  and
      Lee, Yen-Wei  and
      Pai, Ya-Ting  and
      Chang, Wen Yu  and
      Chen, Yun-Nung",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1830/",
    pages = "36711--36756",
    ISBN = "979-8-89176-395-1",
    abstract = "Vision-Language Models (VLMs) often struggle in Taiwanese Mandarin environments due to region-specific orthographic and cultural context. We introduce VisTW, a comprehensive benchmark featuring (i) multiple-choice questions (3,795 academic questions) and (ii) free-form generation evaluation (141 Taiwanese-context free-form pairs). Beyond standard accuracy, we investigate character mixing{---} the unintended production of Simplified Chinese characters under Taiwanese-Mandarin-style prompts{---}and propose a human-grounded purity penalty derived from perceptual thresholds measured from users. Our evaluation reveals substantial character contamination (3{\%}{--}19{\%}) across state-of-the-art VLMs. We find that Gemini-3-Pro significantly outperforms the strongest open-weight baseline, Qwen3 235B MoE, by up to 22 percentage points on dialogue tasks once the purity penalty is applied. These results highlight orthographic consistency as a vital, yet overlooked, dimension for localized multimodal evaluation and deployment."
}