@inproceedings{mamedov-etal-2025-gigachat,
    title = "{G}iga{C}hat Family: Efficient {R}ussian Language Modeling Through Mixture of Experts Architecture",
    author = "Mamedov, Valentin  and
      Kosarev, Evgenii  and
      Leleytner, Gregory  and
      Shchuckin, Ilya  and
      Berezovskiy, Valeriy  and
      Smirnov, Daniil  and
      Kozlov, Dmitry  and
      Averkiev, Sergei  and
      Ivan, Lukyanenko  and
      Proshunin, Aleksandr  and
      Israfilova, Ainur  and
      Baskov, Ivan  and
      Chervyakov, Artem  and
      Shakirov, Emil  and
      Kolesov, Mikhail  and
      Khomich, Daria  and
      Latortseva, Daria  and
      Porkhun, Sergei  and
      Fedorov, Yury  and
      Kutuzov, Oleg  and
      Kudriavtseva, Polina  and
      Soldatova, Sofiia  and
      Egor, Kolodin  and
      Pyatkin, Stanislav  and
      Menshykh, Dzmitry  and
      IUrevich, Grafov Sergei  and
      Damirov, Eldar  and
      Karlov, Vladimir  and
      Gaitukiev, Ruslan  and
      Shatenov, Arkadiy  and
      Fenogenova, Alena  and
      Savushkin, Nikita  and
      Minkin, Fedor",
    editor = "Mishra, Pushkar  and
      Muresan, Smaranda  and
      Yu, Tao",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-demo.10/",
    pages = "93--106",
    ISBN = "979-8-89176-253-4",
    abstract = "Generative large language models (LLMs) have become crucial for modern NLP research and applications across various languages. However, the development of foundational models specifically tailored to the Russian language has been limited, primarily due to the significant computational resources required. This paper introduces the GigaChat family of Russian LLMs, available in various sizes, including base models and instruction-tuned versions. We provide a detailed report on the model architecture, pre-training process, and experiments to guide design choices. In addition, we evaluate their performance on Russian and English benchmarks and compare GigaChat with multilingual analogs. The paper presents a system demonstration of the top-performing models accessible via an API, a Telegram bot, and a Web interface. Furthermore, we have released three open GigaChat models in open-source, aiming to expand NLP research opportunities and support the development of industrial solutions for the Russian language."
}