@inproceedings{kiulian-etal-2025-english,
    title = "From {E}nglish-Centric to Effective Bilingual: {LLM}s with Custom Tokenizers for Underrepresented Languages",
    author = "Kiulian, Artur  and
      Polishko, Anton  and
      Khandoga, Mykola  and
      Kostiuk, Yevhen  and
      Gabrielli, Guillermo  and
      Gaga{\l}a, {\L}ukasz  and
      Zaraket, Fadi  and
      Abu Obaida, Qusai  and
      Garud, Hrishikesh  and
      Wing Yee Mak, Wendy  and
      Chaplynskyi, Dmytro  and
      Amor, Selma  and
      Peradze, Grigol",
    editor = "Romanyshyn, Mariana",
    booktitle = "Proceedings of the Fourth Ukrainian Natural Language Processing Workshop (UNLP 2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria (online)",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/acl25-workshop-ingestion/2025.unlp-1.1/",
    pages = "1--13",
    ISBN = "979-8-89176-269-5",
    abstract = "In this paper, we propose a model-agnostic cost-effective approach to developing bilingual base large language models (LLMs) to support English and any target language. The method includes vocabulary expansion, initialization of new embeddings, model training and evaluation. We performed our experiments with three languages, each using a non-Latin script{---}Ukrainian, Arabic, and Georgian.Our approach demonstrates improved language performance while reducing computational costs. It mitigates the disproportionate penalization of underrepresented languages, promoting fairness and minimizing adverse phenomena such as code-switching and broken grammar. Additionally, we introduce new metrics to evaluate language quality, revealing that vocabulary size significantly impacts the quality of generated text."
}