@inproceedings{goloburda-etal-2025-qorgau,
    title = "Qor{\'{g}}au: Evaluating Safety in {K}azakh-{R}ussian Bilingual Contexts",
    author = "Goloburda, Maiya  and
      Laiyk, Nurkhan  and
      Turmakhan, Diana  and
      Wang, Yuxia  and
      Togmanov, Mukhammed  and
      Mansurov, Jonibek  and
      Sametov, Askhat  and
      Mukhituly, Nurdaulet  and
      Wang, Minghan  and
      Orel, Daniil  and
      Mujahid, Zain Muhammad  and
      Koto, Fajri  and
      Baldwin, Timothy  and
      Nakov, Preslav",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.findings-acl.507/",
    pages = "9765--9784",
    ISBN = "979-8-89176-256-5",
    abstract = "Large language models (LLMs) are known to have the potential to generate harmful content, posing risks to users. While significant progress has been made in developing taxonomies for LLM risks and safety evaluation prompts, most studies have focused on monolingual contexts, primarily in English. However, language- and region-specific risks in bilingual contexts are often overlooked, and core findings can diverge from those in monolingual settings. In this paper, we introduce Qor{\'{g}}au, a novel dataset specifically designed for safety evaluation in Kazakh and Russian, reflecting the unique bilingual context in Kazakhstan, where both Kazakh (a low-resource language) and Russian (a high-resource language) are spoken. Experiments with both multilingual and language-specific LLMs reveal notable differences in safety performance, emphasizing the need for tailored, region-specific datasets to ensure the responsible and safe deployment of LLMs in countries like Kazakhstan. Warning: this paper contains example data that may be offensive, harmful, or biased."
}