@inproceedings{chen-etal-2025-instructioncp,
    title = "{I}nstruction{CP}: A Simple yet Effective Approach for Transferring Large Language Models to Target Languages",
    author = "Chen, Kuang-Ming  and
      Hwang, Jenq-Neng  and
      Lee, Hung-yi",
    editor = "Hahn, Michael  and
      Rani, Priya  and
      Kumar, Ritesh  and
      Shcherbakov, Andreas  and
      Sorokin, Alexey  and
      Serikov, Oleg  and
      Cotterell, Ryan  and
      Vylomova, Ekaterina",
    booktitle = "Proceedings of the 7th Workshop on Research in Computational Linguistic Typology and Multilingual NLP",
    month = aug,
    year = "2025",
    address = "Vinenna. Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/acl25-workshop-ingestion/2025.sigtyp-1.1/",
    pages = "1--6",
    ISBN = "979-8-89176-281-7",
    abstract = "The rapid development of large language models (LLMs) in recent years has largely focused on English, resulting in models that respond exclusively in English. To adapt these models to other languages, continual pre-training (CP) is often employed, followed by supervised fine-tuning (SFT) to maintain conversational abilities. However, CP and SFT can reduce a model{'}s ability to filter harmful content. We propose Instruction Continual Pre-training (InsCP), which integrates instruction tags{---}also known as chat templates{---}into the CP process to prevent loss of conversational proficiency while acquiring new languages. Our experiments demonstrate that InsCP retains conversational and Reinforcement Learning from Human Feedback (RLHF) abilities. Empirical evaluations on language alignment, reliability, and knowledge benchmarks confirm the efficacy of InsCP. Notably, this approach requires only 0.1 billion tokens of high-quality instruction-following data, thereby reducing resource consumption."
}