@inproceedings{dankers-2026-memorisation,
    title = "Memorisation Meets Compositionality in Natural Language Processing",
    author = "Dankers, Verna",
    editor = "Elazar, Yanai  and
      Ettinger, Allyson  and
      Kassner, Nora  and
      Ruder, Sebastian",
    booktitle = "Proceedings of The Big Picture v2: Crafting a Research Narrative",
    month = jul,
    year = "2026",
    address = "San Diego, CA, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.bigpicture-main.12/",
    pages = "144--159",
    ISBN = "979-8-89176-416-3",
    abstract = "Memorisation in deep learning is undergoing a paradigm shift; it is increasingly recognised as a mechanism that can support, rather than hinder, generalisation. This is particularly relevant in NLP, where language combines compositional, generalisable structure with non-compositional expressions such as idioms, requiring memorisation from models and humans alike. My PhD work investigated memorisation in transformer models in generic terms, and through the lens of (non-)compositionality, from both data and model-internal perspectives. I analysed which training examples require memorisation, whether memorisation supports generalisation, and where memorisation occurs within model layers. I also studied how transformers process non-compositional idiom translations and how they balance compositional generalisation with non-compositional memorisation. Based on my findings, I stress that memorisation is an inherent part of learning natural language, can be beneficial, and is partially predictable. Yet it is not cleanly separable from generalisation, both at the level of data and of model parameters. Here, I summarise those findings and reflect on my PhD work."
}