Fangyin Ma
2026
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs
Wenhao Yu | Zhicong Lu | Bo Lv | Fangyin Ma | Kaiwen Wei | Shihao Yang | Nayu Liu
Findings of the Association for Computational Linguistics: ACL 2026
Wenhao Yu | Zhicong Lu | Bo Lv | Fangyin Ma | Kaiwen Wei | Shihao Yang | Nayu Liu
Findings of the Association for Computational Linguistics: ACL 2026
Knowledge editing aims to efficiently update factual information in Large Language Models (LLMs) without full retraining. However, existing methods still suffer from performance degradation in batch knowledge editing. We identify that semantic representation entanglement, such as overlapping concepts and shared syntactic patterns, accumulates interference in the representation space and reduces editing precision. To bridge this gap, in this paper, we propose Orthogonal Representation Editing (ORE), which performs edits in the hidden representation space of LLMs by constructing a general semantic subspace and enforcing orthogonal constraints on edit vectors, effectively decoupling semantic entanglement. Furthermore, we introduce a gated non-linear representation head to enable adaptive learning of editing locations and precise control over knowledge injection. Extensive experiments show that ORE outperforms existing methods and achieves superior performance in cross-lingual knowledge editing scenarios. We release our code at https://github.com/YVVH/ORE.