Linguistic Neuron Overlap Patterns to Facilitate Cross-lingual Transfer on Low-resource Languages

Yuemei Xu, Kexin Xu, Jian Zhou, Ling Hu, Lin Gui


Abstract
The current Large Language Models (LLMs) face significant challenges in improving their performance on low-resource languagesand urgently need data-efficient methods without costly fine-tuning.From the perspective of language-bridge,we propose a simple yet effective method, namely BridgeX-ICL, to improve the zero-shot Cross-lingual In-Context Learning (X-ICL) for low-resource languages. Unlike existing works focusing on language-specific neurons,BridgeX-ICL explores whether sharingneurons can improve cross-lingual performance in LLMs.We construct neuron probe data from the ground-truth MUSE bilingual dictionaries, and define a subset of language overlap neurons accordingly to ensure full activation of these anchored neurons.Subsequently, we propose an HSIC-based metric to quantify LLMs’ internal linguistic spectrumbased on overlapping neurons, guiding optimal bridge selection.The experiments conducted on 4 cross-lingual tasks and 15 language pairs from 7diverse families, covering both high-low and moderate-low pairs, validate the effectiveness of BridgeX-ICL and offer empirical insights into the underlying multilingual mechanisms of LLMs. The code is publicly available at https://github.com/xuyuemei/BridgeX-ICL.
Anthology ID:
2025.emnlp-main.1407
Volume:
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
27646–27661
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1407/
DOI:
Bibkey:
Cite (ACL):
Yuemei Xu, Kexin Xu, Jian Zhou, Ling Hu, and Lin Gui. 2025. Linguistic Neuron Overlap Patterns to Facilitate Cross-lingual Transfer on Low-resource Languages. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27646–27661, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Linguistic Neuron Overlap Patterns to Facilitate Cross-lingual Transfer on Low-resource Languages (Xu et al., EMNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1407.pdf
Checklist:
 2025.emnlp-main.1407.checklist.pdf