Ilya Shchuckin
2025
GigaChat Family: Efficient Russian Language Modeling Through Mixture of Experts Architecture
Valentin Mamedov | Evgenii Kosarev | Gregory Leleytner | Ilya Shchuckin | Valeriy Berezovskiy | Daniil Smirnov | Dmitry Kozlov | Sergei Averkiev | Lukyanenko Ivan | Aleksandr Proshunin | Ainur Israfilova | Ivan Baskov | Artem Chervyakov | Emil Shakirov | Mikhail Kolesov | Daria Khomich | Daria Latortseva | Sergei Porkhun | Yury Fedorov | Oleg Kutuzov | Polina Kudriavtseva | Sofiia Soldatova | Kolodin Egor | Stanislav Pyatkin | Dzmitry Menshykh | Grafov Sergei IUrevich | Eldar Damirov | Vladimir Karlov | Ruslan Gaitukiev | Arkadiy Shatenov | Alena Fenogenova | Nikita Savushkin | Fedor Minkin
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
Valentin Mamedov | Evgenii Kosarev | Gregory Leleytner | Ilya Shchuckin | Valeriy Berezovskiy | Daniil Smirnov | Dmitry Kozlov | Sergei Averkiev | Lukyanenko Ivan | Aleksandr Proshunin | Ainur Israfilova | Ivan Baskov | Artem Chervyakov | Emil Shakirov | Mikhail Kolesov | Daria Khomich | Daria Latortseva | Sergei Porkhun | Yury Fedorov | Oleg Kutuzov | Polina Kudriavtseva | Sofiia Soldatova | Kolodin Egor | Stanislav Pyatkin | Dzmitry Menshykh | Grafov Sergei IUrevich | Eldar Damirov | Vladimir Karlov | Ruslan Gaitukiev | Arkadiy Shatenov | Alena Fenogenova | Nikita Savushkin | Fedor Minkin
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
Generative large language models (LLMs) have become crucial for modern NLP research and applications across various languages. However, the development of foundational models specifically tailored to the Russian language has been limited, primarily due to the significant computational resources required. This paper introduces the GigaChat family of Russian LLMs, available in various sizes, including base models and instruction-tuned versions. We provide a detailed report on the model architecture, pre-training process, and experiments to guide design choices. In addition, we evaluate their performance on Russian and English benchmarks and compare GigaChat with multilingual analogs. The paper presents a system demonstration of the top-performing models accessible via an API, a Telegram bot, and a Web interface. Furthermore, we have released three open GigaChat models in open-source, aiming to expand NLP research opportunities and support the development of industrial solutions for the Russian language.
Search
Fix author
Co-authors
- Sergei Averkiev 1
- Ivan Baskov 1
- Valeriy Berezovskiy 1
- Artem Chervyakov 1
- Eldar Damirov 1
- Kolodin Egor 1
- Yury Fedorov 1
- Alena Fenogenova 1
- Ruslan Gaitukiev 1
- Grafov Sergei IUrevich 1
- Ainur Israfilova 1
- Lukyanenko Ivan 1
- Vladimir Karlov 1
- Daria Khomich 1
- Mikhail Kolesov 1
- Evgenii Kosarev 1
- Dmitry Kozlov 1
- Polina Kudriavtseva 1
- Oleg Kutuzov 1
- Daria Latortseva 1
- Gregory Leleytner 1
- Valentin Mamedov 1
- Dzmitry Menshykh 1
- Fedor Minkin 1
- Sergei Porkhun 1
- Aleksandr Proshunin 1
- Stanislav Pyatkin 1
- Nikita Savushkin 1
- Emil Shakirov 1
- Arkadiy Shatenov 1
- Daniil Smirnov 1
- Sofiia Soldatova 1
Venues
- acl1