Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage

Hanyin Shao, Jie Huang, Shen Zheng, Kevin Chang

Abstract
The advancement of large language models (LLMs) brings notable improvements across various applications, while simultaneously raising concerns about potential private data exposure. One notable capability of LLMs is their ability to form associations between different pieces of information, but this raises concerns when it comes to personally identifiable information (PII). This paper delves into the association capabilities of language models, aiming to uncover the factors that influence their proficiency in associating information. Our study reveals that as models scale up, their capacity to associate entities/information intensifies, particularly when target pairs demonstrate shorter co-occurrence distances or higher co-occurrence frequencies. However, there is a distinct performance gap when associating commonsense knowledge versus PII, with the latter showing lower accuracy. Despite the proportion of accurately predicted PII being relatively small, LLMs still demonstrate the capability to predict specific instances of email addresses and phone numbers when provided with appropriate prompts. These findings underscore the potential risk to PII confidentiality posed by the evolving capabilities of LLMs, especially as they continue to expand in scale and power.
Anthology ID:
2024.findings-eacl.54
Volume:
Findings of the Association for Computational Linguistics: EACL 2024
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
814–825
Language:
URL:
https://aclanthology.org/2024.findings-eacl.54
DOI:
Bibkey:
Cite (ACL):
Hanyin Shao, Jie Huang, Shen Zheng, and Kevin Chang. 2024. Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage. In Findings of the Association for Computational Linguistics: EACL 2024, pages 814–825, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage (Shao et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp-22-attachments/2024.findings-eacl.54.pdf
PDF Search