@inproceedings{wang-etal-2026-speechllm,
    title = "{S}peech{LLM}-as-Judges: Towards General and Interpretable Speech Quality Evaluation",
    author = "Wang, Hui  and
      Zhao, Jinghua  and
      Yang, Yifan  and
      Liu, Shujie  and
      Chen, Junyang  and
      Zhang, Yanzhe  and
      Zhao, Shiwan  and
      Li, Jinyu  and
      Zhou, Jiaming  and
      Sun, Haoqin  and
      Lu, Yan  and
      Qin, Yong",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.349/",
    pages = "7675--7700",
    ISBN = "979-8-89176-390-6",
    abstract = "Generative speech technologies are progressing rapidly, but evaluating the perceptual quality of synthetic speech remains a core challenge. Existing methods typically rely on scalar scores or binary decisions, which lack interpretability and generalization across tasks and languages. We present SpeechLLM-as-Judges, a new paradigm for enabling large language models (LLMs) to conduct structured and explanation-based speech quality evaluation. To support this direction, we introduce SpeechEval, a large-scale dataset containing 32,207 multilingual speech clips and 128,754 annotations spanning four tasks: quality assessment, pairwise comparison, improvement suggestion, and deepfake detection. Based on this resource, we develop SQ-LLM, a speech-quality-aware LLM trained with chain-of-thought reasoning and reward optimization to improve capability. Experimental results show that SQ-LLM delivers strong performance across tasks and languages, revealing the potential of this paradigm for advancing speech quality evaluation. The relevant code, models, and data are publicly available at https://github.com/NKU-HLT/SpeechLLM-as-Judges."
}