@inproceedings{li-etal-2026-srdetection,
    title = "{S}r{D}etection: A Self-Referential Framework for Data Leakage Detection in Code Large Language Models",
    author = "Li, Shuaimin  and
      Fan, Liyang  and
      li, Zeyang  and
      Wan, Zhuoyue  and
      Lin, Yufang  and
      Ni, Shiwen  and
      Fang, Feiteng  and
      Alinejad-Rokny, Hamid  and
      Song, Yuanfeng  and
      Jing, Kun  and
      Zhang, Chen Jason  and
      Yang, Min",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.252/",
    pages = "5117--5129",
    ISBN = "979-8-89176-395-1",
    abstract = "Evaluating code large language models (Code LLMs) requires reliable detection of data leakage, where benchmark performance is artificially inflated by exposure to benchmark data during pre-training. Existing approaches either assume access to proprietary training corpora, rely on brittle heuristics such as timestamp filtering, or use external reference sets with manually tuned, non-generalizable thresholds. To address these limitations, we introduce \textbf{SrDetection}, a unified \textbf{s}elf-\textbf{r}eferential leakage detection framework for both gray-box (access to model logits) and black-box (access to model outputs) settings. SrDetection generates semantically equivalent variants of a benchmark sample and detects leakage by contrasting the model{'}s behavior on the original versus its variants, flagging cases where the original is disproportionately easier for the model. We further design a controlled leakage detection testbed and evaluate SrDetection in this environment. Across different models and training stages, SrDetection improves average F1 by 21.52 points in the gray-box setting and 14.46 points in the black-box setting over strong baselines, demonstrating robust, threshold-independent leakage detection. Finally, a gray-box study of 15 widely used Code LLMs on four popular benchmarks reveals benchmark-specific leakage patterns beyond prior overlap-based analyses[Source code and data are available at {\ensuremath{<}}https://github.com/SMinL/SrDetectionCode{\ensuremath{>}}]."
}