Abdelaziz Bounhar
2025
Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts
Guokan Shang
|
Hadi Abdine
|
Ahmad Chamma
|
Amr Mohamed
|
Mohamed Anwar
|
Abdelaziz Bounhar
|
Omar El Herraoui
|
Preslav Nakov
|
Michalis Vazirgiannis
|
Eric P. Xing
Proceedings of The Third Arabic Natural Language Processing Conference
We introduce Nile-Chat-4B, 3x4B-A6B, and 12B, a collection of LLMs for Egyptian dialect, uniquely designed to understand and generate texts written in both Arabic and Latin scripts. Specifically, with Nile-Chat-3x4B-A6B, we introduce a novel language adaptation approach by leveraging the Branch-Train-MiX strategy to merge script-specialized experts, into a single MoE model. Our Nile-Chat models significantly outperform leading multilingual and Arabic LLMs, such as LLaMa, Jais, and ALLaM, on our newly introduced Egyptian evaluation benchmarks, which span both understanding and generative tasks. Notably, our 12B model delivers a 14.4% performance gain over Qwen2.5-14B-Instruct on Latin-script benchmarks. All our resources are publicly available. We believe this work presents a comprehensive methodology for adapting LLMs to a single language with dual-script usage, addressing an often overlooked aspect in contemporary LLM development.
Search
Fix author
Co-authors
- Hadi Abdine 1
- Mohamed Anwar 1
- Ahmad Chamma 1
- Omar El Herraoui 1
- Amr Mohamed 1
- show all...