@inproceedings{liu-etal-2025-relation-specific,
    title = "On Relation-Specific Neurons in Large Language Models",
    author = "Liu, Yihong  and
      Chen, Runsheng  and
      Hirlimann, Lea  and
      Hakimi, Ahmad Dawar  and
      Wang, Mingyang  and
      Kargaran, Amir Hossein  and
      Rothe, Sascha  and
      Yvon, Fran{\c{c}}ois  and
      Schuetze, Hinrich",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/lei-li-partial-disambiguation/2025.emnlp-main.52/",
    pages = "992--1022",
    ISBN = "979-8-89176-332-6",
    abstract = "In large language models (LLMs), certain neurons can store distinct pieces of knowledge learned during pretraining. While factual knowledge typically appears as a combination of relations and entities, it remains unclear whether some neurons focus on a relation itself {--} independent of any entity. We hypothesize such neurons detect a relation in the input text and guide generation involving such a relation. To investigate this, we study the LLama-2 family on a chosen set of relations, with a statistics-based method. Our experiments demonstrate the existence of relation-specific neurons. We measure the effect of selectively deactivating candidate neurons specific to relation $r$ on the LLM{'}s ability to handle (1) facts involving relation $r$ and (2) facts involving a different relation $r' \neq r$. With respect to their capacity for encoding relation information, we give evidence for the following three properties of relation-specific neurons. $\textbf{(i) Neuron cumulativity.}$ Multiple neurons jointly contribute to processing facts involving relation $r$, with no single neuron fully encoding a fact in $r$ on its own. $\textbf{(ii) Neuron versatility.}$ Neurons can be shared across multiple closely related as well as less related relations. In addition, some relation neurons transfer across languages. $\textbf{(iii) Neuron interference.}$ Deactivating neurons specific to one relation can improve LLMs' factual recall performance for facts of other relations. We make our code and data publicly available at \url{https://github.com/cisnlp/relation-specific-neurons}."
}