@inproceedings{feng-etal-2026-mimiclm,
    title = "{M}imic{LM}: Zero-Shot Voice Imitation through Autoregressive Modeling of Pseudo-Parallel Speech Corpora",
    author = "Feng, Tao  and
      Wang, Yuxiang  and
      Wang, Yuancheng  and
      Zhang, Xueyao  and
      Chen, Dekun  and
      Wang, Chaoren  and
      Guan, Xun  and
      Wu, Zhizheng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1020/",
    pages = "20392--20405",
    ISBN = "979-8-89176-395-1",
    abstract = "Voice imitation aims to transform *source* speech to match a *reference* speaker{'}s timbre and speaking style while preserving linguistic content. A straightforward approach is to train on triplets of *(source, reference, target)*, where *source* and *target* share the same content but *target* matches the *reference*{'}s voice characteristics, yet such data is extremely scarce. Existing approaches either employ carefully designed disentanglement architectures to bypass this data scarcity or leverage external systems to synthesize pseudo-parallel training data. However, the former requires intricate model design, and the latter faces a quality ceiling when synthetic speech is used as training *targets*. To address these limitations, we propose MimicLM, which takes a novel approach by using synthetic speech as training *sources* while retaining real recordings as *targets*. This design enables the model to learn directly from real speech distributions, breaking the synthetic quality ceiling. Building on this data construction approach, we incorporate interleaved text-audio modeling to guide the generation of content-accurate speech and apply post-training with preference alignment to mitigate the inherent distributional mismatch when training on synthetic data. Experiments demonstrate that MimicLM achieves superior voice imitation quality with a simple yet effective architecture, significantly outperforming existing methods in naturalness while maintaining competitive similarity scores across speaker identity, accent, and emotion dimensions."
}