HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection

Luke S. Patterson, Li Wang, Adam Faulkner


Abstract
Thanks to the rapid adoption of AI code assistants powered by large language models (LLMs), industry codebases are, increasingly, a hybrid of AI- and human-authored code. For risk management and productivity analysis purposes, it is crucial to enable fine-grained location detection of AI-generated code. To develop algorithms for this task, quality benchmarks are needed to assess performance. However, existing benchmarks tend to comprise academic, LeetCode-style problems and presume a code snippet is either completely human-authored or completely AI-authored, which is not reflective of the diverse intents and styles of industry codebases utilizing AI code assistants. To fill these gaps, we introduce HybridCodeAuthorship, a novel benchmark of Python code files with interleaved human- and AI-authored lines of code to simulate authentic utilization of AI code assistants. In this paper, we first present our dataset construction pipeline, which leverages CodeSearchNet, a massive collection of links to open sourced repositories on GitHub. We then benchmark the performance of two state-of-the-art AI-generated code detection algorithms at both the line- and chunk-level. Experimental results demonstrate that HybridCodeAuthorship is a challenging benchmark with a top-scoring algorithm, AIGCode Detector, obtaining a highest F1 score of 0.48 and 0.56 on line-level and chunk-level code detection tasks, respectively.
Anthology ID:
2026.lrec-main.117
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
1520–1532
Language:
URL:
https://preview.aclanthology.org/ingest-lrec/2026.lrec-main.117/
DOI:
Bibkey:
Cite (ACL):
Luke S. Patterson, Li Wang, and Adam Faulkner. 2026. HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection. International Conference on Language Resources and Evaluation, main:1520–1532.
Cite (Informal):
HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection (Patterson et al., LREC 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-lrec/2026.lrec-main.117.pdf