@inproceedings{flemings-etal-2025-estimating,
    title = "Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Models",
    author = "Flemings, James  and
      Jiang, Bo  and
      Zhang, Wanrong  and
      Takhirov, Zafar  and
      Annavaram, Murali",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1220/",
    pages = "25092--25108",
    ISBN = "979-8-89176-251-0",
    abstract = "Language models (LMs) rely on their parametric knowledge augmented with relevant contextual knowledge for certain tasks, such as question answering. However, the contextual knowledge can contain private information that may be leaked when answering queries, and estimating this privacy leakage is not well understood. A straightforward approach of directly comparing an LM{'}s output to the contexts can overestimate the privacy risk, since the LM{'}s parametric knowledge might already contain the augmented contextual knowledge. To this end, we introduce \textit{context influence}, a metric that builds on differential privacy, a widely-adopted privacy notion, to estimate the privacy leakage of contextual knowledge during decoding. Our approach effectively measures how each subset of the context influences an LM{'}s response while separating the specific parametric knowledge of the LM. Using our context influence metric, we demonstrate that context privacy leakage occurs when contextual knowledge is out of distribution with respect to parametric knowledge. Moreover, we experimentally demonstrate how context influence properly attributes the privacy leakage to augmented contexts, and we evaluate how factors{--} such as model size, context size, generation position, etc.{--} affect context privacy leakage. The practical implications of our results will inform practitioners of the privacy risk associated with augmented contextual knowledge."
}