\documentclass[11pt]{article}

% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
%\usepackage[review]{acl}
\usepackage{acl}
\usepackage{times}
\usepackage{latexsym}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage{tabularx}
\usepackage{graphicx}
% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}
% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets

% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

% This is also not strictly necessary, and may be commented out.
% However, it will improve the aesthetics of text in
% the typewriter font.
\usepackage{inconsolata}

\title{SSN-CSE-CODECATALYSTS at SemEval-2026 Task 13: Integrating Transformer Semantics and AST-Derived Structural Features for AI-Generated Code Detection.}

\author{Bhuvana J\textsuperscript{1}, {\bf Ramanan Mahendran}\textsuperscript{1}, {\bf Siddharth Chandrasekar S}\textsuperscript{1} \\
{\bf Pragatheesh J}\textsuperscript{1}, {\bf Rethanya P}\textsuperscript{1} \\
\vspace{0.2cm} % Optional: adds a little breathing room before the affiliation
\textsuperscript{1}Sri Sivasubramaniya Nadar College of Engineering, Chennai, India}
%  \textsuperscript{1}Affiliation 1,
%  \textsuperscript{2}Affiliation 2,
%  \textsuperscript{3}Affiliation 3,
%  \textsuperscript{4}Affiliation 4,
%  \textsuperscript{5}Affiliation 5
%\\
%  \small{
%    \textbf{Correspondence:} \href{mailto:email@domain}{email@domain}
%  }
%}
\begin{document}
\maketitle
\begin{abstract}
Pre-trained transformers often struggle with multi-lingual code classification due to sequence length constraints and difficulties in explicitly capturing deep structural complexities. To address this for SemEval Task 13 \citep{orel-etal-2026-semeval-2026}, a hybrid neural architecture that fuses CodeBERT's semantic embeddings is proposed. Handcrafted software engineering metrics is presented, with a Head+Tail truncation strategy to preserve crucial logic in long sequences while simultaneously extracting explicit Abstract Syntax Tree (AST) features via tree-sitter—including maximum depth, branching factor, and cyclomatic complexity. By integrating dense language model representations with explicit structural heuristics, this work provides a robust and scalable solution for enhanced code classification.
\end{abstract}

\section{Introduction}

Source code classification is a fundamental task in automated software engineering, increasingly addressed using Natural Language Processing (NLP) techniques as multi-lingual codebases grow in scale and diversity. Pre-trained transformer models such as CodeBERT and RoBERTa \citep{liu2019roberta} learn rich semantic representations of source code, but real-world deployment exposes key limitations, including strict input sequence length constraints and weak modeling of explicit hierarchical structure. Transformers process code as linear token sequences, capturing semantic relationships while failing to encode deterministic structural rules, and the standard 512-token limit often truncates large files, removing critical control-flow logic. SemEval-2026 Task 13 motivates robust multi-lingual solutions to these challenges, leading to a late-fusion architecture that integrates CodeBERT embeddings with deterministic structural features extracted via \texttt{tree-sitter}, including AST depth, branching factor, cyclomatic complexity, nesting depth, and lexical ratios. A Head+Tail truncation strategy preserves structurally salient code regions, enabling effective fusion of semantic and structural signals across diverse programming languages.

\section{Background}

SemEval Task 13 addresses the critical challenge of distinguishing human-written source code from snippets generated by Large Language Models (LLMs) by requiring binary classification into Human (0) or Machine (1) labels. The task places a specific emphasis on Out-Of-Distribution (OOD) generalization, providing training data in C++, Python, and Java, while challenging models with unseen languages like Go, PHP, C#, C, and JavaScript, as well as unseen domains involving research and deployed code. Evaluation is primarily measured using the Macro F1-score as the lead leaderboard metric. Although prior work in code classification has been dominated by pre-trained Transformer models such as the bimodal NL-PL representations of CodeBERT \citep{feng-etal-2020-codebert}, the structural data-flow integration of GraphCodeBERT \citep{guo-etal-2021-graphcodebert}, the layer-wise analysis of EL-CodeBERT \citep{liu-etal-2022-elcodebert}, and the prompt-based learning of CodePrompt (2024), this method shifts focus toward injecting domain-specific software metrics directly into the decision function. By utilizing a late-fusion design, dense CodeBERT semantic vectors are combined with discrete Abstract Syntax Tree (AST) features, such as nesting depth and branching factors, alongside statistical heuristics like cyclomatic complexity. This integration of learned semantic embeddings with deterministic code-quality features aims to improve robustness when semantic signals degrade in unseen languages, aligning with the rigorous OOD requirements of the SemEval-2026 Task 13 evaluation.

\section{System Overview}

The proposed system is designed to address challenges in detecting machine-generated code in multi-lingual, out-of-distribution (OOD) settings. Large Language Models often produce syntactically valid but structurally generic code, whereas human-written code exhibits distinctive nesting and complexity patterns. Standard transformer models, constrained by sequence length limits, frequently truncate the sections of code where these signatures occur.

To address this limitation, a Late-Fusion Hybrid Architecture is introduced that combines dense semantic representations from a pre-trained transformer with explicit, handcrafted software engineering metrics as shown in Figure~\ref{fig:architecture}.
\begin{figure*}
    \centering
    \includegraphics[width=1\linewidth]{image.png}
    \caption{Architectural Overview of the HybridCode Framework}
    \label{fig:architecture}
\end{figure*}

\subsection{Feature Extraction Strategy}

The model assumes that human and machine code differ in graph complexity and lexical density. To capture this, a feature engineering pipeline extracts a 7-dimensional vector, $v_{stat} \in \mathbb{R}^7$, for each code sample.

\paragraph{A. Abstract Syntax Tree (AST) Analysis}

The model uses Tree-Sitter \citep{tree-sitter} to parse raw code into its constituent AST. Unlike regex-based heuristics, this enables measurement of true topological code complexity. 

\begin{itemize}
    \item \textbf{Max Depth ($d_{max}$):} The longest path from the root to a leaf node, serving as a proxy for algorithmic complexity.
    \item \textbf{Average Branching Factor ($b_{avg}$):} The ratio of total child nodes to non-leaf nodes, capturing logical ``width''.
\end{itemize}

\paragraph{B. Cyclomatic Complexity ($v_{cc}$)}
Since full control-flow graph construction is computationally expensive at scale, Cyclomatic Complexity \citep{mccabe1976cyclomatic} is approximated by counting branching keywords:

\begin{equation}
v_{cc} = \sum_{k \in K} \text{count}(k) + 1
\end{equation}

Where $K = \{ \text{'if', 'for', 'while', 'case', 'catch', '\&\&', '||', '?' } \}$.

\paragraph{C. Lexical \& Nesting Metrics}
\begin{itemize}
    \item \textbf{Nesting Depth ($v_{nd}$):} The maximum depth of nested blocks (defined by \{\} or ()), capturing the ``spaghetti code'' tendency often found in human submissions.
    \item \textbf{Lexical Ratios:} The approach computes the average identifier length ($l_{id}$), comment-to-code ratio ($r_{com}$), and keyword-to-identifier ratio ($r_{kw}$) to encode stylistic signatures.
\end{itemize}
\subsection{Head+Tail Truncation Strategy}
A significant challenge in this task is the 512-token limit of BERT-based models. Standard ``head-only'' truncation discards the end of the file, which often contains critical return statements or closing logic. A \textit{Head+Tail Truncation} strategy is implemented to preserve the distinct structural boundaries of the code.

Given a tokenized sequence $T$ of length $L > 510$, the truncated sequence $T_{trunc}$ is defined as:
\begin{equation}
    T_{trunc} = [T_{0}, \dots, T_{254}] \oplus [T_{L-255}, \dots, T_{L-1}]
\end{equation}

The pseudocode for the implementation is provided in Algorithm \ref{alg:truncation}.

\begin{algorithm}
\caption{Head+Tail Truncation Strategy}
\label{alg:truncation}
\begin{algorithmic}[1]
\Procedure{ApplyTruncation}{$tokens, max\_len=512$}
    \State $special\_tokens \gets 2$ \Comment{[CLS] and [SEP]}
    \State $capacity \gets max\_len - special\_tokens$
    \If{$length(tokens) \le capacity$}
        \State \Return $[CLS] + tokens + [SEP] + Padding$
    \EndIf
    \State $half \gets capacity // 2$
    \State $head \gets tokens[0 : half]$
    \State $tail \gets tokens[length(tokens) - half : length(tokens)]$
    \State \Return $[CLS] + head + tail + [SEP]$
\EndProcedure
\end{algorithmic}
\end{algorithm}

\subsection{Hybrid Neural Architecture}
This classification model is a late-fusion network that integrates three distinct feature modalities.

This approach employs CodeBERT (\texttt{microsoft/codebert-base}) as the primary encoder. CodeBERT is a bimodal transformer model pre-trained on the CodeSearchNet dataset. For an input sequence $X$, the model outputs a sequence of hidden states. The embedding of the first token (\texttt{[CLS]}) is utilized as the aggregate semantic representation of the code:
\begin{equation}
    v_{bert} = \text{CodeBERT}(X)_{[CLS]} \in \mathbb{R}^{768}
\end{equation}

\textbf{Language Embedding}
To improve generalization across the diverse language set (Python, C++, Java, Go, PHP, etc.),a learnable language embedding layer is introduced. The language label $L$ is mapped to a dense vector:
\begin{equation}
    v_{lang} = \text{Embedding}(L) \in \mathbb{R}^{16}
\end{equation}
This allows the network to condition its structural expectations (e.g., ``braces are common in Java but not Python'') on the specific language context.

\textbf{Late Fusion \& Classification Head}
The final representation $v_{final}$ is constructed by concatenating the semantic, language, and statistical vectors:
\begin{equation}
    v_{final} = \text{Concat}(v_{bert}, v_{lang}, v_{stat}) \in \mathbb{R}^{768 + 16 + 7}
\end{equation}

This fused vector is passed through a Multi-Layer Perceptron (MLP) designed with Layer Normalization to stabilize the disparate feature scales:
\begin{itemize}
    \item \textbf{Linear Projection:} $h_1 = W_1 v_{final} + b_1$ (Output dim: 256)
    \item \textbf{Normalization \& Activation:} $h_2 = \text{ReLU}(\text{LayerNorm}(h_1))$
    \item \textbf{Regularization:} $h_3 = \text{Dropout}(h_2, p=0.2)$
    \item \textbf{Bottleneck:} $h_4 = \text{ReLU}(W_2 h_3 + b_2)$ (Output dim: 64)
    \item \textbf{Logits:} $y = W_3 h_4 + b_3$ (Output dim: 2)
\end{itemize}

\subsection{Concrete Execution Example}
To illustrate the pipeline, consider a C++ snippet where \texttt{tree-sitter} detects a max nesting of 2 and a cyclomatic complexity of 2. 

\begin{enumerate}
    \item \textbf{Feature Extraction:} Vector $v_{stat} = [2.0, 1.0, 3.5, 0.2, \dots]$ is computed.
    \item \textbf{Forward Pass:} The snippet generates $v_{bert}$ (768-d) and $v_{lang}$ (16-d for C++).
    \item \textbf{Fusion:} These are concatenated into a 791-d vector for final classification.
\end{enumerate}

\subsection{Resources \& Optimization}
This approach utilizes pre-compiled \texttt{tree-sitter} binaries and \texttt{microsoft/codebert-base} weights. To manage computational costs, \textbf{Mixed Precision Training (AMP)} with \texttt{bfloat16} and \textbf{Gradient Checkpointing} is employed, reducing VRAM usage by 60\% and allowing a batch size of 160 on a single NVIDIA RTX A4000 GPU.

\section{Experimental Setup}
\subsection{Data Splitting and Utilization}
The official SemEval-2026 Task 13 dataset \citep{orel2026aicdbenchchallengingbenchmark} provided is utilized in Parquet format. The data was split into three distinct sets to ensure robust evaluation:
\begin{itemize}
    \item \textbf{Training Set:} The approach utilized the full \texttt{train.parquet} dataset, consisting of 500,000 labeled code samples, to fine-tune the model parameters.
    \item \textbf{Validation Set:} A separate \texttt{validation.parquet} set of 100,000 samples was used for hyperparameter tuning and model checkpointing. 
    \item \textbf{Test Set:} The final performance was evaluated on \texttt{test.parquet}, consisting of 1,000 unseen samples, covering both in-distribution and out-of-distribution (OOD) programming languages.
\end{itemize}

\subsection{Preprocessing and Feature Engineering}
Raw source code underwent a preprocessing pipeline:
\begin{itemize}
    \item \textbf{Text Normalization:} Code samples were processed with \texttt{normalize\_whitespace} to collapse redundant spaces and newlines into single spaces, lowering token count while preserving semantics.
    \item \textbf{Tokenization and Truncation:} \texttt{RobertaTokenizer} from \texttt{microsoft/codebert-base} was applied. To respect the 512-token limit, a custom Head+Tail strategy retained the first 254 and last 254 tokens (plus \texttt{[CLS]} and \texttt{[SEP]}), discarding the middle segment to preserve initialization and return logic.
    \item \textbf{Handcrafted Feature Scaling:} A 7-dimensional structural feature vector (e.g., AST depth, cyclomatic complexity) was normalized with \texttt{torch.log1p} to reduce outlier and skew impact before concatenation with neural embeddings.
\end{itemize}

\subsection{Implementation Details}
Experiments used PyTorch and Hugging Face Transformers, with random seed 42 for Python, NumPy, and PyTorch.

\paragraph{Hardware and Optimization:}
Training ran on a single NVIDIA RTX A4000 GPU. To increase throughput, Mixed Precision Training (\texttt{torch.amp}, \texttt{bfloat16}) and Gradient Checkpointing on the CodeBERT backbone were enabled, reducing VRAM usage and permitting a batch size of 160.

\paragraph{Hyperparameter Configuration:} 
The model was trained for 3 epochs using the parameters detailed in Table \ref{tab:hyperparams}.

\begin{table}[h]
\centering
\small
% tabularx ensures the table fits exactly within the column width
\begin{tabularx}{\columnwidth}{llX}
\hline
\textbf{Hyperparameter} & \textbf{Value} & \textbf{Rationale} \\ \hline
Batch Size & 160 & Maximized for 16GB VRAM w/ grad. checkpointing. \\
Learning Rate & $5 \times 10^{-5}$ & Peak LR for the OneCycle scheduler. \\
Optimizer & AdamW & Used with \texttt{fused=True} for faster computation. \\
Scheduler & OneCycleLR & Linear warmup followed by cosine annealing. \\
Weight Decay & 0.01 & Standard regularization for AdamW. \\
Dropout & 0.2 & Applied to the classification head. \\ \hline
\end{tabularx}
\caption{Hyperparameter configuration for model training.}
\label{tab:hyperparams}
\end{table}

\subsection{Evaluation Measures}
The primary evaluation metric for this task is the Macro F1-Score. We computed this metric at the end of each training epoch on the validation set to monitor convergence and select the optimal model checkpoint.
\begin{equation}
\text{Macro F1} = \frac{1}{N} \sum_{i=1}^{N} \frac{2 \times P_i \times R_i}{P_i + R_i}
\end{equation}
Where $P_i$ and $R_i$ are the precision and recall for class $i$, and $N$ is the number of classes (2: Human vs. Machine).


\section{Results}

\subsection{Main Quantitative Findings}
The proposed system was evaluated on the official SemEval-2026 Task 13 (Subtask A) blind test set, achieving a \textbf{Macro F1-score of 0.32826} and ranking \textbf{68th} on the global leaderboard. The reported score corresponds to the best-performing configuration among three submitted attempts. Despite successful end-to-end inference, the results reveal a substantial performance gap relative to leading submissions, primarily due to limited generalization to Out-Of-Distribution programming languages in the test set.

\subsection{Quantitative Analysis and Ablations}
A sharp contrast exists between validation (In-Distribution) and official test (OOD) performance, as summarized in Table \ref{tab:results_comparison}.

\begin{table}[h]
\centering
\small
\begin{tabularx}{\columnwidth}{lXX}
\hline
\textbf{Split} & \textbf{Macro F1} & \textbf{Nature of Data} \\ \hline
Validation & 0.9882 & In-Distribution (C++, Java, Python) \\
Official Test & 0.3283 & OOD (Go, PHP, C\#, etc.) \\ \hline
\end{tabularx}
\caption{Comparison of performance between In-Distribution and OOD sets.}
\label{tab:results_comparison}
\end{table}

\paragraph{In-Distribution (Validation) Performance:}
The Late-Fusion architecture achieved a peak Macro F1-score of 0.9882 at Epoch 3, indicating effective separation of human-written and machine-generated code within known programming languages.

\paragraph{Out-Of-Distribution (Test) Performance:}
Performance declined to 0.328 on the official test set, indicating limited robustness to unseen languages. Reliance on language-specific components, including the \textit{Language Embedding} layer and \texttt{tree-sitter} AST parsers, caused brittleness.

\paragraph{Ablation Insight:}
The Structural Pathway (AST features) was sensitive to parser availability. For supported languages, depth and branching statistics provided discriminative signals. For unsupported languages, null or zero-valued AST outputs were treated as class-specific patterns, causing systematic misclassification.

\subsection{Error Analysis}
Prediction behavior reveals the following failure modes:

\begin{itemize}
    \item \textbf{``Zero-Feature'' Trap:} For OOD languages, structural metrics often defaulted to zero, causing the MLP head to overfit in-distribution complexity ranges and reducing Recall.
    
    \item \textbf{Confusion Matrix Bias:} The low F1-score reflects a skewed Confusion Matrix with a high False Negative rate under OOD conditions.
    
    \item \textbf{Truncation Artifacts:} Head+Tail truncation preserved boundary logic but introduced semantic drift due to language-dependent verbosity.
\end{itemize}

Overall, near-perfect in-distribution performance ($F1 \approx 0.99$) contrasted with limited zero-shot generalization, as rigid structural metrics degraded robustness in unseen languages.
\section{Conclusion}

\textit{HybridCode} was presented for SemEval-2026 Task 13 (Subtask A) to distinguish human-written and machine-generated source code. The proposed system employed a late-fusion architecture combining CodeBERT semantic embeddings with handcrafted structural features derived from \texttt{tree-sitter} AST analysis, supported by a Head+Tail truncation strategy to preserve salient code regions. While strong in-distribution performance was observed (Macro F1-score of 0.9882), the proposed system ranked 68 on the official leaderboard with a test F1-score of 0.32826, revealing a substantial generalization gap. Explicit structural features proved highly discriminative for known languages but reduced robustness in out-of-distribution settings where parsers failed or returned null outputs, indicating that rigid structural priors hinder multilingual generalization.


\section*{Acknowledgments}

Appreciation is extended to the Department of Computer Science and Engineering at Sri Sivasubramaniya Nadar College of Engineering, Chennai, India, for providing computational resources and academic support.

\bibliography{custom,anthology-1}
\end{document}
