@inproceedings{wang-etal-2026-interpretable,
    title = "Interpretable Safety Alignment via {SAE}-Constructed Low-Rank Subspace Adaptation",
    author = "Wang, Dianyun  and
      Ma, Qingsen  and
      Shang, Yuhu  and
      Lu, Zhifeng  and
      Xu, Zhenbo  and
      Ning, Lechen  and
      Wu, Huijia  and
      He, Zhaofeng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.215/",
    pages = "4703--4721",
    ISBN = "979-8-89176-390-6",
    abstract = "Safety alignment{---}training large language models (LLMs) to refuse harmful requests while remaining helpful{---}is critical for responsible deployment. Prior work established that safety behaviors are governed by low-rank structures, suggesting parameter-efficient fine-tuning (PEFT) should be well-suited for alignment. However, Low-Rank Adaptation (LoRA) consistently underperforms full fine-tuning and reinforcement learning on safety benchmarks. We attribute this gap to semantic entanglement: safety-relevant directions are intertwined with unrelated concepts due to polysemanticity, impeding implicit subspace identification. To address this, we propose SAILS (Safety Alignment via Interpretable Low-rank Subspace), which leverages Sparse Autoencoders (SAEs) to disentangle representations into monosemantic features, constructs an interpretable safety subspace from SAE decoder directions, and uses it to initialize LoRA adapters. Theoretically, we prove that SAE-based identification achieves arbitrarily small recovery error under monosemanticity assumptions, while direct identification suffers an irreducible error floor. Empirically, SAILS achieves up to 99.6{\%} safety rates across multiple model families and scales, exceeding full fine-tuning and matching RLHF-based models, with only 0.2{\%} of parameters updated and providing interpretability."
}