@inproceedings{rakshit-goldberg-2025-meaning,
    title = "Meaning-infused grammar: Gradient Acceptability Shapes the Geometric Representations of Constructions in {LLM}s",
    author = "Rakshit, Supantho  and
      Goldberg, Adele E.",
    editor = "Bonial, Claire  and
      Torgbi, Melissa  and
      Weissweiler, Leonie  and
      Blodgett, Austin  and
      Beuls, Katrien  and
      Van Eecke, Paul  and
      Tayyar Madabushi, Harish",
    booktitle = "Proceedings of the Second International Workshop on Construction Grammars and NLP",
    month = sep,
    year = "2025",
    address = {D{\"u}sseldorf, Germany},
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/iwcs-25-ingestion/2025.cxgsnlp-1.15/",
    pages = "151--157",
    ISBN = "979-8-89176-318-0",
    abstract = "The usage-based constructionist (UCx) approach to language posits that language comprises a network of learned form-meaning pairings (\textit{constructions}) whose use is largely determined by their meanings or functions, requiring them to be graded and probabilistic. This study investigates whether the internal representations in Large Language Models (LLMs) reflect the proposed function-infused gradience. We analyze representations of the English Double Object (DO) and Prepositional Object (PO) constructions in Pythia-1.4B, using a dataset of 5000 sentence pairs systematically varied by human-rated preference strength for DO or PO. Geometric analyses show that the separability between the two constructions' representations, as measured by energy distance or Jensen-Shannon divergence, is systematically modulated by gradient preference strength, which depends on lexical and functional properties of sentences. That is, more prototypical exemplars of each construction occupy more distinct regions in activation space, compared to sentences that could have equally well have occured in either construction. These results provide evidence that LLMs learn rich, meaning-infused, graded representations of constructions and offer support for geometric measures for representations in LLMs."
}