WWTC@UniA at SemEval-2026 Task 13: BERT-based Code Authorship Detection and Qualitative Analysis

Linda Kupfer, Lisa Hader, Christian Jaumann, Annemarie Friedrich


Abstract
This paper describes our system for SemEval-2026 Task 13 on detecting machine-generated code. We fine-tune small encoder-only models for detecting human-written versus machine-generated code and for identifying which large language model (LLM) family was used to obtain code. We find that a strong, general-purpose model (ModernBERT) outperforms models specifically pre-trained for the code domain. In the official evaluation, our system ranks 5th on subtask B and 6th on subtask C. Our detailed analysis reveals that comments and other natural language text that is part of the code snippets provide valuable information for identifying the LLM family that generated it. Moreover, we show that the embeddings of our finetuned ModernBERT do not distinguish well between LLM families, but they cluster human-written code by programming language.
Anthology ID:
2026.semeval-1.298
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2359–2375
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.298/
DOI:
Bibkey:
Cite (ACL):
Linda Kupfer, Lisa Hader, Christian Jaumann, and Annemarie Friedrich. 2026. WWTC@UniA at SemEval-2026 Task 13: BERT-based Code Authorship Detection and Qualitative Analysis. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 2359–2375, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
WWTC@UniA at SemEval-2026 Task 13: BERT-based Code Authorship Detection and Qualitative Analysis (Kupfer et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.298.pdf
Supplementarymaterial:
 2026.semeval-1.298.SupplementaryMaterial.zip