@inproceedings{zaghouani-etal-2026-albanianllmsafety,
    title = "{A}lbanian{LLMS}afety: A Safety Evaluation Dataset for Large Language Models in {A}lbanian",
    author = "Zaghouani, Wajdi  and
      Aldous, Kholoud Khalil  and
      Fejzullaj, Isra",
    editor = "Ojha, Atul Kr.  and
      Sakti, Sakriani  and
      Soria, Claudia  and
      Melero, Maite  and
      McCrae, John P.  and
      Lignos, Constantine  and
      Liu, Chao-Hong  and
      Claramunt, German Rigau  and
      Rehm, Georg",
    booktitle = "Proceedings of the {SIGUL} 2026 Joint Workshop with {ELE}, {EURALI}, and {DCLRL}: Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered Languages",
    month = may,
    year = "2026",
    address = "Palma, Mallorca, Spain",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/revision-workflow/2026.sigul-1.7/",
    doi = "10.63317/42i5uiy6yyqq",
    pages = "66--73",
    abstract = "Safety evaluation of Large Language Models (LLMs) has largely focused on high-resource languages, leaving low-resource languages critically underserved. We present AlbanianLLMSafety, the first publicly available safety evaluation dataset for LLMs in Albanian, a linguistically distinct low-resource language with approximately 7.5 million speakers across Albania, Kosovo, North Macedonia, and the diaspora. The dataset contains 2,951 prompts spanning 11 safety categories, including self-harm, violence, racist content, child exploitation, and radicalization, with an average of 268 prompts per category. Each prompt is provided in Albanian with an English reference translation and a detailed category label. This resource addresses a significant gap in safety evaluation infrastructure for low-resource languages and provides an essential benchmark for developing safer, more inclusive LLMs. The dataset will be provided upon request to support safety evaluation, fine-tuning, red-teaming, and guardrail development for Albanian-speaking communities."
}