@inproceedings{kassem-etal-2025-alpaca,
    title = "{ALPACA} {AGAINST} {VICUNA}: Using {LLM}s to Uncover Memorization of {LLM}s",
    author = "Kassem, Aly M.  and
      Mahmoud, Omar  and
      Mireshghallah, Niloofar  and
      Kim, Hyunwoo  and
      Tsvetkov, Yulia  and
      Choi, Yejin  and
      Saad, Sherif  and
      Rana, Santu",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.naacl-long.421/",
    pages = "8296--8321",
    ISBN = "979-8-89176-189-6",
    abstract = "In this paper, we investigate the overlooked impact of instruction-tuning on memorization in large language models (LLMs), which has largely been studied in base, pre-trained models. We propose a black-box prompt optimization method where an attacker LLM agent uncovers higher levels of memorization in a victim agent, surpassing traditional approaches that prompt the model directly with training data. Using an iterative rejection-sampling process, we design instruction-based prompts that minimize overlap with training data to avoid providing direct solutions while maximizing overlap between the victim{'}s output and the training data to induce memorization. Our method shows 23.7{\%} more overlap with training data compared to state-of-the-art baselines. We explore two attack settings: an analytical approach that determines the empirical upper bound of the attack, both with and without access to responses for prompt initialization, and a practical classifier-based method for assessing memorization without access to memorized data. Our findings reveal that instruction-tuned models can expose pre-training data as much as, or more than, base models; contexts beyond the original training data can lead to leakage; and instructions generated by other LLMs open new avenues for automated attacks, which we believe require further exploration."
}