Layer-wise Swapping for Generalizable Multilingual Safety

Hyunseo Shin, Wonseok Hwang


Abstract
Despite the rapid advancements of Large Language Models (LLMs), safety risks remain a critical challenge for low-resource languages. Existing safety datasets are predominantly English-centric, limiting progress in multilingual safety alignment. As a result, low-resource expert models—fine-tuned on their respective instruction datasets—tend to exhibit higher unsafety rates compared to their high-resource counterparts. In this work, we propose a safety aware layer swapping method that transfers safety alignment from an English safety expert to low-resource language experts without additional training. To further enhance transfer ability, our method adaptively selects or blends modules based on their degree of specialization. Our approach preserves performance on general language understanding tasks while enhancing safety in the target languages. Experimental results show that the proposed method achieves comparable performance to the language expert on general benchmarks such as MMMLU, BELEBELE, and MGSM, while producing more aligned and less harmful responses on the MultiJail safety benchmark
Anthology ID:
2026.eacl-long.98
Volume:
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Demberg, Kentaro Inui, Lluís Marquez
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2223–2238
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.98/
DOI:
Bibkey:
Cite (ACL):
Hyunseo Shin and Wonseok Hwang. 2026. Layer-wise Swapping for Generalizable Multilingual Safety. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2223–2238, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
Layer-wise Swapping for Generalizable Multilingual Safety (Shin & Hwang, EACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.98.pdf