@inproceedings{dhar-etal-2026-evaluating,
    title = "Evaluating Adjective-Noun Compositionality in {LLM}s: Functional vs Representational Perspectives",
    author = "Dhar, Ruchira  and
      Peng, Qiwei  and
      S{\o}gaard, Anders",
    editor = "Mohammad, Saif M.  and
      Ousidhoum, Nedjma",
    booktitle = "Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*{SEM} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.starsem-conference.8/",
    pages = "125--135",
    ISBN = "979-8-89176-413-2",
    abstract = "Compositionality is considered central to language abilities. As performant language systems, how do large language models (LLMs) do on compositional tasks? We evaluate adjective{--}noun compositionality in LLMs using two complementary setups: prompt-based functional assessment and a representational analysis of internal model states. Our results reveal a striking divergence between task performance and internal states. While LLMs reliably develop compositional representations, they fail to translate consistently into functional task success across model variants. Consequently, we highlight the importance of contrastive evaluation for obtaining a more complete understanding of model capabilities."
}