@inproceedings{zhou-etal-2026-efficient,
    title = "Efficient Training for Cross-lingual Speech Language Models",
    author = "Zhou, Yan  and
      Fang, Qingkai  and
      Hong, Yun  and
      Feng, Yang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.642/",
    pages = "13155--13167",
    ISBN = "979-8-89176-395-1",
    abstract = "Currently, large language models (LLMs) predominantly focus on the text modality. To enable more natural human-AI interaction, speech LLMs are emerging, but building effective end-to-end speech LLMs remains challenging due to limited data and the difficulty in expanding to more languages. In this paper, we introduce $\textbf{C}$ross-lingual $\textbf{S}$peech $\textbf{L}$anguage $\textbf{M}$odel ($\textbf{CSLM}$), an efficient training method for cross-lingual speech LLMs based on discrete speech tokens. We propose a novel alignment strategy that achieves cross-modal and cross-lingual alignment through continual pre-training. By conducting instruction fine-tuning following a speech-text interleaved chain-of-modality generation process, we enhance modal alignment at a finer granularity, thereby improving generation quality and reducing latency. CSLM aligns different modalities and languages simultaneously without the need for massive speech data, thus exhibiting good language scalability. Evaluations on cross-modal tasks, mono-lingual conversational tasks, and cross-lingual conversational tasks demonstrate CSLM{'}s strong cross-modal alignment capabilities and general task abilities."
}