@inproceedings{almeida-etal-2026-conceptual,
    title = "Conceptual Hierarchies within {LLM}s",
    author = "Almeida, Tiago  and
      Zhu, Zining  and
      Ning, Yue",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.2041/",
    pages = "41067--41079",
    ISBN = "979-8-89176-395-1",
    abstract = "While it is widely agreed that large language models (LLMs) store concepts from multiple semantic hierarchies, much remains unknown regarding the structure of this storage. The correspondence between the functional roles of LLM components and the semantic hierarchies of knowledge remains underexplored in the current literature. For example, is information organized hierarchically within sections of an LLM? We take an initial step towards causally examining the correspondence between hierarchical concepts and the multi-granular structures (layers and attention heads) of various models. Specifically, we generate a dataset of semantic hierarchies and investigate their storage locations in six LLMs using activation patching, a causal intervention technique. At the layer level, our findings show a moderate indication that concepts at finer levels of granularity are stored around 61-78{\%} of the time ($p$ {\ensuremath{<}} 0.01) before those at coarser granularity. There is evidence for this trend at the attention level; however, the high variability in attention level results suggests that concepts are stored across attention heads rather than within. Our results offer insight into semantic organization within LLMs."
}