@inproceedings{hu-etal-2026-vcb,
    title = "{VCB} Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents",
    author = "Hu, Jiliang  and
      Wang, Wenfu  and
      Li, Zuchao  and
      Li, Chenxing  and
      Zhao, Yiyang  and
      Li, Hanzhao  and
      Zhang, Liqiang  and
      Yu, Meng  and
      Yu, Dong",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1659/",
    pages = "33176--33200",
    ISBN = "979-8-89176-395-1",
    abstract = "While large audio language models (LALMs) have driven significant progress in multimodal conversational systems, current benchmarks suffer from critical limitations: they are largely English-centric, use synthetic speech, and fail to provide comprehensive, discriminative evaluation across key dimensions. To fill this gap, we present Voice Chat Bot Bench (VCB Bench), a novel, high-quality Chinese benchmark built exclusively on real human speech. VCB Bench assesses LALMs across three complementary axes: instruction following (including speech-level control beyond text commands), knowledge understanding (including general knowledge, reasoning, and daily dialogue), and robustness (evaluating stability under variations in content, environment, and speaker characteristics). Experiments conducted on representative LALMs reveal notable performance disparities and offer tangible insights for future improvements. VCB Bench serves as a reproducible and fine-grained framework, providing standardized evaluation and practical guidance for the development of Chinese voice conversational models."
}