MindFlayer at SemEval-2026 Task 13:LACR-ENS: Calibration-Aware Ensemble Routing for Cross-Language AI-Generated Code Detection

Jerin Romijah Tuli, Talukder Naemul Hasan Naem, Md. Sartaj Alam Pritom


Abstract
This paper presents LACR-ENS, a calibration-aware ensemble system for detecting AI-generated code across eight programming languages (SemEval-2026 Task 13). We identify a severe asymmetric out-of-distribution (OOD) failure in fine-tuned code transformers: Expected Calibration Error doubles from 0.09 (seen languages) to 0.18 (unseen languages), and high-confidence predictions (p0.80) are wrong 39% of the time on OOD inputs. We propose Language-Aware Confidence Routing (LACR), formally equivalent to implicit per-language temperature scaling, which reduces OOD ECE to 0.11 and improves macro-F1 by +0.013 over fixed-threshold ensembling. A language-family proximity analysis reveals that syntactic distance to training languages predicts OOD F1 with Pearson r=+0.94, providing a principled, label-free signal for deployment risk assessment and motivating a continuous routing extension. Our system combines UniXCoder and GraphCodeBERT via weighted logit-level fusion and achieves macro-F1 0.538 , outperforming comparable encoder-only systems. We additionally document a HuggingFace label inversion pitfall that suppressed our initial score by approximately 0.29 F1.
Anthology ID:
2026.semeval-1.294
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2322–2329
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.294/
DOI:
Bibkey:
Cite (ACL):
Jerin Romijah Tuli, Talukder Naemul Hasan Naem, and Md. Sartaj Alam Pritom. 2026. MindFlayer at SemEval-2026 Task 13:LACR-ENS: Calibration-Aware Ensemble Routing for Cross-Language AI-Generated Code Detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 2322–2329, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
MindFlayer at SemEval-2026 Task 13:LACR-ENS: Calibration-Aware Ensemble Routing for Cross-Language AI-Generated Code Detection (Tuli et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.294.pdf
Supplementarymaterial:
 2026.semeval-1.294.SupplementaryMaterial.zip