Safeguarding Privacy of Retrieval Data against Membership Inference Attacks: Is This Query Too Close to Home?

Yujin Choi, Youngjoo Park, Junyoung Byun, Jaewook Lee, Jinseong Park


Abstract
Retrieval-augmented generation (RAG) mitigates the hallucination problem in large language models (LLMs) and has proven effective for personalized usages. However, delivering private retrieved documents directly to LLMs introduces vulnerability to membership inference attacks (MIAs), which try to determine whether the target data point exists in the private external database or not. Based on the insight that MIA queries typically exhibit high similarity to only one target document, we introduce a novel similarity-based MIA detection framework designed for the RAG system. With the proposed method, we show that a simple detect-and-hide strategy can successfully obfuscate attackers, maintain data utility, and remain system-agnostic against MIA. We experimentally prove its detection and defense against various state-of-the-art MIA methods and its adaptability to existing RAG systems.
Anthology ID:
2025.findings-emnlp.438
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2025
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8241–8258
Language:
URL:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.438/
DOI:
10.18653/v1/2025.findings-emnlp.438
Bibkey:
Cite (ACL):
Yujin Choi, Youngjoo Park, Junyoung Byun, Jaewook Lee, and Jinseong Park. 2025. Safeguarding Privacy of Retrieval Data against Membership Inference Attacks: Is This Query Too Close to Home?. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 8241–8258, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Safeguarding Privacy of Retrieval Data against Membership Inference Attacks: Is This Query Too Close to Home? (Choi et al., Findings 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.438.pdf
Checklist:
 2025.findings-emnlp.438.checklist.pdf