@inproceedings{wang-etal-2026-batonvoice,
    title = "{B}aton{V}oice: An Operationalist Framework for Enhancing Controllable Speech Synthesis with Linguistic Intelligence from {LLM}s",
    author = "Wang, Yue  and
      Ma, Ruotian  and
      Chen, Xingyu  and
      Shi, Zhengliang  and
      Yang, Morunliu  and
      Chen, Wanshun  and
      Liu, Huang  and
      Yao, Jiadi  and
      He, Xin  and
      Yang, Qu  and
      Jiang, Qingxuan  and
      Ye, Fanghua  and
      Li, Juntao  and
      Tu, Zhaopeng  and
      Li, Xiaolong  and
      Bo, Liefeng  and
      Zhang, Min",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.2165/",
    pages = "46683--46697",
    ISBN = "979-8-89176-390-6",
    abstract = "The rise of Large Language Models (LLMs) is reshaping multimodel models, with speech synthesis being a prominent application. However, existing approaches often underutilize the linguistic intelligence of these models, typically failing to leverage their powerful instruction-following capabilities. This limitation hinders the model{'}s ability to follow text instructions for controllable Text-to-Speech{~}(TTS). To address this, we propose a new paradigm inspired by operationalism that decouples instruction understanding from speech generation. We introduce BatonVoice, a framework where an LLM acts as a conductor, understanding user instructions and generating a textual plan {--} explicit vocal features (e.g., pitch, energy). A separate TTS model, the orchestra, then generates the speech from these features. To realize this component, we develop BatonTTS, a TTS model trained specifically for this task. Our experiments demonstrate that BatonVoice achieves strong performance in controllable and emotional speech synthesis, outperforming strong open- and closed-source baselines. Notably, our approach enables remarkable zero-shot cross-lingual generalization, accurately applying feature control abilities to languages unseen during post-training. This demonstrates that objectifying speech into textual vocal features can more effectively unlock the linguistic intelligence of LLMs."
}