@inproceedings{mondal-etal-2025-language,
    title = "Language-Specific Neurons Do Not Facilitate Cross-Lingual Transfer",
    author = "Mondal, Soumen Kumar  and
      Sen, Sayambhu  and
      Singhania, Abhishek  and
      Jyothi, Preethi",
    editor = "Drozd, Aleksandr  and
      Sedoc, Jo{\~a}o  and
      Tafreshi, Shabnam  and
      Akula, Arjun  and
      Shu, Raphael",
    booktitle = "The Sixth Workshop on Insights from Negative Results in NLP",
    month = may,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.insights-1.6/",
    pages = "46--62",
    ISBN = "979-8-89176-240-4",
    abstract = "Multilingual large language models (LLMs) aim towards robust natural language understanding across diverse languages, yet their performance significantly degrades on low-resource languages. This work explores whether existing techniques to identify language-specific neurons can be leveraged to enhance cross-lingual task performance of low-resource languages. We conduct detailed experiments covering existing language-specific neuron identification techniques (such as LanguageActivation Probability Entropy and activation probability-based thresholding) andneuron-specific LoRA fine-tuning with models like Llama 3.1 and Mistral Nemo. We find that such neuron-specific interventions are insufficient to yield cross-lingual improvements on downstream tasks (XNLI, XQuAD) in low-resource languages. This study highlights the challenges in achieving cross-lingual generalization and provides critical insights for multilingual LLMs."
}