@article{aizawa-etal-2026-building,
    title = "Building Effective {J}apanese Medical {LLM}s with an Open Recipe for Domain Adaptation through Continued Pre-training",
    author = "Aizawa, Akiko  and
      Arase, Yuki  and
      Cheng, Fei  and
      Huang, Jiahao  and
      Huang, Zhiyi  and
      Jiang, Junfeng  and
      Kanazawa, Teruhito  and
      Kawahara, Daisuke  and
      Kobayashi, Kazuma  and
      Kodama, Takashi  and
      Kurohashi, Sadao  and
      Oda, Yusuke  and
      Tsuta, Yuma  and
      Wan, Zhen  and
      Yang, Zhishen  and
      Yokota, Rio",
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    journal = "International Conference on Language Resources and Evaluation",
    volume = "main",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://preview.aclanthology.org/ingest-lrec/2026.lrec-main.817/",
    pages = "10405--10423",
    abstract = "In high-stakes domains such as medicine, ensuring transparency of the training corpus is essential, with careful consideration of local healthcare landscapes; however, the majority of existing medical large language models (LLMs) have not disclosed the details of their training corpora. Here, we introduce an open recipe for domain adaptation of LLMs to the Japanese medical domain. We employed fully open-source Japanese general-domain LLMs as base models, whose pre-training datasets are also disclosed. To establish effective corpora for domain adaptation through continued pre-training, we started with small-scale medical datasets and ultimately constructed a medical corpus consisting of 79.6B tokens, incorporating local clinical guidelines, medical textbooks, and other domain-specific resources. The resulting LLM from continued pre-training, namely SIP-med-llm-8x13B, with an active parameter count of 22B, demonstrated favorable accuracy on benchmarks including the Japanese National Medical Examination. This performance was comparable to that of 70B-parameter open-weight models whose construction details remain non-transparent. This represents the first case in the Japanese medical field where complete corpus details have been disclosed for fully from-scratch development, providing important insights for future efforts to construct medical LLMs tailored to the specific characteristics of local contexts. The model is available publicly at this Hugging Face repository: https://huggingface.co/SIP-med-LLM/SIP-jmed-llm-2-8x13b-OP-instruct."
}