@inproceedings{zheng-etal-2025-voicecraft,
    title = "{V}oice{C}raft-{X}: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing",
    author = "Zheng, Zhisheng  and
      Peng, Puyuan  and
      Diwan, Anuj  and
      Huynh, Cong Phuoc  and
      Sun, Xiaohang  and
      Liu, Zhu  and
      Bhat, Vimal  and
      Harwath, David",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.137/",
    pages = "2737--2756",
    ISBN = "979-8-89176-332-6",
    abstract = "We introduce VoiceCraft-X, an autoregressive neural codec language model which unifies multilingual speech editing and zero-shot text-to-speech (TTS) synthesis across 11 languages: English, Mandarin, Korean, Japanese, Spanish, French, German, Dutch, Italian, Portuguese, and Polish. VoiceCraft-X utilizes the Qwen3 large language model for phoneme-free cross-lingual text processing and a novel token reordering mechanism with time-aligned text and speech tokens to handle both tasks as a single sequence generation problem. The model generates high-quality, natural-sounding speech, seamlessly creating new audio or editing existing recordings within one framework. VoiceCraft-X shows robust performance in diverse linguistic settings, even with limited per-language data, underscoring the power of unified autoregressive approaches for advancing complex, real-world multilingual speech applications. Audio samples are available at https://zhishengzheng.com/voicecraft-x/."
}