\begin{table}[ht]
    \centering
    \small
    \begin{tabular}{p{0.25\linewidth} p{0.65\linewidth}}
        \hline
        \textbf{Aspect} & \textbf{Description} \\
        \hline
        Source & Reddit submission statements (first-level comments by discussion starters). \\
        Availability & Train/dev (redacted) on Zenodo (DOI: 10.5281/zenodo.15114171); plain text obtained by rehydration (Arctic Shift API). \\
        Train size & 4,361 instances (after rehydration and preprocessing). \\
        Instance format & Per instance: \texttt{\_id}, \texttt{text} (rehydrated, preprocessed), \texttt{conspiracy}, \texttt{markers}, \texttt{subreddit}, \texttt{annotator}. \\
        Preprocessing & Markdown $\rightarrow$ plain text, URL replacement, strip ``submission statement''/``ss'' prefix; character indices in \texttt{markers} refer to this preprocessed text. \\
        Splits & $\sim$90\% released for training/development; test set held privately for CodaBench evaluation. \\
        Annotation & Conspiracy labels: Krippendorff's $\alpha = 0.58$ (binary); $\alpha = 0.50$ including ``Can't tell''. Oversampling of r/conspiracy (1,150 comments); rest from other subreddits. \\
        \hline
    \end{tabular}
    \caption{Overview of the PsyCoMark dataset.}
    \label{tab:dataset_summary}
\end{table}

\begin{table}[ht]
    \centering
    \small
    \begin{tabular}{p{0.25\linewidth} p{0.65\linewidth}}
        \hline
        \textbf{Metric} & \textbf{Use \& Definition} \\
        \hline
        Weighted F1 & \textbf{Primary;} official CodaBench metric for both detection and extraction. F1 per class, averaged with class support weights. \\
        Macro F1 & Secondary; unweighted mean of per-class F1. \\
        Accuracy & Fraction of correct predictions. \\
 (\textit{Yes}/\textit{No}). \\
        \hline
    \end{tabular}
    \caption{Evaluation metrics for Conspiracy Detection and Extraction.}
    \label{tab:det_metrics}
\end{table}


Where to add this: 
\subsection{Data Ingestion and Preprocessing}
To systematically capture the linguistic nuances of conspiracy discourse, we developed a comprehensive EDA and feature extraction pipeline. Because the primary task formulation is binary classification, we explicitly configured this pipeline to isolate instances labeled as \textit{Yes} (conspiracy) or \textit{No} (non-conspiracy), omitting ambiguous \textit{Can't tell} samples. The ingestion module standardizes heterogeneous labels into a binary schema, resolves duplicate conflicts via unique indexing, and processes the raw text into semantic and syntactic features.



\subsubsection{Psycholinguistic Feature Extraction}
We construct a multidimensional linguistic profile for each text by extracting features across four primary domains:


 \textbf{Lexical Complexity and Diversity:} We measure vocabulary richness using Type-Token Ratio (TTR), Hapax Legomena ratio, average word length, and Shannon entropy. To mitigate the impact of varying text lengths, we compute advanced moving-window metrics: Moving-Average TTR (MATTR) and the Measure of Textual Lexical Diversity (MTLD).


 \textbf{Readability:} Utilizing \texttt{textstat}, we calculate a suite of readability indices---including Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog, SMOG, and the Automated Readability Index (ARI)---to capture the cognitive load required to process the text.


 \textbf{Syntax and Discourse:} We extract fine-grained POS distribution ratios. Furthermore, we deploy heuristic pattern matching to quantify the usage of discourse markers, epistemic hedges, certainty markers, and the passive voice.


 \textbf{Sentiment and Emotion:} We apply VADER sentiment analysis to compute continuous compound sentiment scores, capturing the text's affective tone, supplemented by emotion dimensions where applicable. (\autoref{fig:eda})

\subsubsection{Semantic, Statistical, and Baseline Modeling}
Beyond surface-level features, we capture deep semantic representations using the SentenceTransformer model \texttt{all-mpnet-base-v2}. We project these dense embeddings into a lower-dimensional space using UMAP and cluster them via HDBSCAN to identify distinct narrative groupings. In parallel, we apply Latent Dirichlet Allocation (LDA) and BERTopic to extract latent thematic structures.

We conduct univariate independent $t$-tests on all numerical features to isolate statistically significant differences between the \textit{Yes} and \textit{No} classes. Twelve features reached significance at $p < 0.05$ (see Table~\ref{tab:univariate-tests}). Conspiracy texts exhibit higher usage of stopwords, adverbs, question marks, and pronouns. (\autoref{tab:univariate-tests} ) We utilize these features to train a baseline Logistic Regression classifier, which attains a mean F1 of approximately \textbf{0.36--0.41} (5-fold CV). Applying TreeSHAP to a Random Forest model corroborates these univariate findings, highlighting the specific syntactic cues that best separate the classes.


\subsection{Machine Learning}
To systematically evaluate the predictive power of our generated psycholinguistic metrics, we developed a standardized modeling pipeline that aggregates the full set of 62 EDA outputs into a single design matrix. To guarantee rigorous cross-model comparisons, we enforce a globally shared data split. We isolate a stratified validation set of precisely 100 documents prior to training, mirroring the SemEval evaluation environment. Model performance is comprehensively tracked via precision, recall, and F1 scores (\autoref{tab:ml_results}) highlighting the correlation between human-interpretable linguistic features and conspiracy discourse.

\subsection{Transformer-Based Architectures}
We experimented with five distinct transformer-based configurations, progressing from standard fine-tuning to data-augmented parameter-efficient approaches. \autoref{tab:transformer_results} summarizes the performance across these models.


 \textbf{Baseline Transformer Classifiers:} We trained \texttt{distilbert-base-uncased} (full fine-tuning) and \texttt{bert-base-uncased} (top 10 layers un-frozen) on the original binary dataset without data augmentation. 


 \textbf{Data-Augmented RoBERTa Architectures:} To combat class imbalance, we applied English--German Backtranslation alongside synonym replacement. We trained two variations of \texttt{roberta-large} utilizing a 2$\times$ augmented dataset (top 12 layers un-frozen) and a 3$\times$ augmented dataset (top 18 layers un-frozen).


 \textbf{Parameter-Efficient Fine-Tuning (LoRA):} Our highest-performing deep learning strategy utilized the rehydrated competition splits (\texttt{train\_rehydrated} and \texttt{dev\_rehydrated}). We applied LoRA to a frozen \texttt{roberta-large} backbone, injecting adapters ($r=16, \alpha=32$, dropout=$0.1$) exclusively into the \textit{query}, \textit{value}, \textit{key}, and \textit{dense} modules. This configuration enabled the training of a highly complex model with minimal memory overhead, yielding our best overall results. The model performed at 0.79 F1 in phase 1 of evaluation, while the model was further improved later on scoring 0.81 F1.  (\autoref{tab:transformer_results})


% Conspiracy Markers Extraction — refined for paper
% Uses: train_rehydrated.jsonl; RoBERTa-large + LoRA; 90/10 split; CodaBench Overlap F1

\subsection{Conspiracy Markers Extraction Architecture}

We address \textbf{span extraction}: for each document, the system predicts a set of character-level spans and assigns each span to one of five target types (\textit{Actor}, \textit{Action}, \textit{Victim}, \textit{Evidence}, \textit{Effect}). Each predicted span is represented by \texttt{startIndex}, \texttt{endIndex}, and \texttt{type}. We compare two token-classification setups, both built on \textbf{RoBERTa-large} and fine-tuned with LoRA (rank $r=16$, $\alpha=32$, applied to \texttt{query}, \texttt{value}, \texttt{key}, and \texttt{dense} projections).

\subsubsection{Dataset and Splits}

Training relies on \texttt{train\_rehydrated.jsonl}, which provides \texttt{text} and \texttt{markers} (character-offset span annotations). We split the rehydrated data randomly (seed 42) into 90\% training and 10\% validation; the split is not stratified by marker type. The official dev and test sets are reserved exclusively for inference and evaluation. Predictions are formatted as JSONL with \texttt{\_id} and \texttt{markers} for CodaBench submission.

\subsubsection{Models and Baselines}


 \textbf{Five-model ensemble:} We train one token-classifier per marker type, each with a 3-label BIO scheme ($O$, $B$-type, $I$-type) for that type only. The five models are trained separately; at inference we merge their predictions per document. We apply class weighting (higher weight on $B$/$I$ vs.\ $O$) to mitigate label imbalance. This setup achieves approximately 0.15 Overlap F1 on the dev set.\\

 \textbf{Single multi-type model:} We train a single RoBERTa-large token-classifier with a joint BIO scheme over all five types (11 labels: $O$ plus $B$/$I$ for each of Actor, Action, Victim, Evidence, Effect). Training uses early stopping on validation entity F1 (micro F1 over non-$O$ labels) with patience 3. \autoref{tab:extraction_comparison} shows the results achieved by using both of the techniques.





\begin{figure*}[t]
\centering
\resizebox{\textwidth}{!}{%
\begin{tikzpicture}[
    node distance=1.0cm and 1.6cm,
    every node/.style={font=\small},
    block/.style={rectangle, draw, rounded corners, align=center, minimum width=3.2cm, minimum height=0.85cm},
    smallblock/.style={rectangle, draw, rounded corners, align=center, minimum width=2.8cm, minimum height=0.75cm},
    arrow/.style={-Latex, thick}
]

% Top input
\node[block] (input) {Raw Reddit Comment};

% =========================
% Detection branch (left)
% =========================
\node[block, below left=1.2cm and 4.0cm of input] (det_head) {\textbf{Detection Branch}};

\node[smallblock, below=0.8cm of det_head] (feat) {Feature Extraction\\(62D)};
\node[smallblock, below=0.8cm of feat] (ml) {ML Classifiers\\(LR, SVM, RF, XGB, DT, MLP)};
\node[smallblock, right=1.8cm of feat] (det_trf) {Transformer Encoder\\(DistilBERT/BERT)};
\node[smallblock, below=0.8cm of det_trf] (det_cls) {Classification Head};
\node[block, below=1.0cm of ml, xshift=1.9cm] (det_out) {Conspiracy Label\\(Yes / No)};

% =========================
% Extraction branch (right)
% =========================
\node[block, below right=1.2cm and 4.0cm of input] (ext_head) {\textbf{Extraction Branch}};

\node[smallblock, below=0.8cm of ext_head] (tok) {Tokenizer\\+ Offset Mapping};
\node[smallblock, below=0.8cm of tok] (ext_trf) {RoBERTa-large + LoRA};
\node[smallblock, below=0.8cm of ext_trf] (bio) {Token Classification\\(BIO Tags)};
\node[smallblock, below=0.8cm of bio] (recon) {Span Reconstruction\\(Char Offsets)};
\node[block, below=0.8cm of recon] (ext_out) {Marker Spans\\Actor, Action, Victim,\\Evidence, Effect};

% =========================
% Shared arrows from input
% =========================
\draw[arrow] (input) -- (det_head);
\draw[arrow] (input) -- (ext_head);

% Detection arrows
\draw[arrow] (det_head) -- (feat);
\draw[arrow] (feat) -- (ml);
\draw[arrow] (ml) -- (det_out);

\draw[arrow] (det_head) -- (det_trf);
\draw[arrow] (det_trf) -- (det_cls);
\draw[arrow] (det_cls) -- (det_out);

% Extraction arrows
\draw[arrow] (ext_head) -- (tok);
\draw[arrow] (tok) -- (ext_trf);
\draw[arrow] (ext_trf) -- (bio);
\draw[arrow] (bio) -- (recon);
\draw[arrow] (recon) -- (ext_out);

\end{tikzpicture}%
}
\caption{Hierarchical architecture of our two-branch framework: document-level conspiracy detection (left) and character-level psycholinguistic marker extraction (right).}
\label{fig:architecture}
\end{figure*}




Table~\ref{tab:per_role_extraction} reports per-role marker extraction performance under token-overlap matching (IoU $\geq$ 0.5). For this post-hoc analysis, we generated predictions on \texttt{dev\_rehydrated} and compared them against the publicly released gold annotations in \texttt{dev\_public}. This analysis is intended to diagnose role-specific behavior and should be interpreted separately from official blind test evaluation.
\begin{table}[t]
\centering
\small
\resizebox{\columnwidth}{!}{%
\begin{tabular}{lcccccc}
\toprule
\textbf{Role} & \textbf{Precision} & \textbf{Recall} & \textbf{F1} & \textbf{TP} & \textbf{Pred} & \textbf{Gold} \\
\midrule
Action         & 0.1207 & 0.2692 & 0.1667 & 28 & 232 & 104 \\
Actor          & 0.2245 & 0.7279 & 0.3432 & 99 & 441 & 136 \\
Effect         & 0.0806 & 0.1408 & 0.1026 & 10 & 124 & 71 \\
Evidence       & 0.0658 & 0.1370 & 0.0889 & 10 & 152 & 73 \\
Victim         & 0.1659 & 0.5139 & 0.2508 & 37 & 223 & 72 \\
\midrule
\textbf{Micro Average} & 0.1570 & 0.4035 & 0.2260 & 184 & 1172 & 456 \\
\bottomrule
\end{tabular}%
}
\caption{Per-role token-overlap extraction results at IoU $\geq$ 0.5 for \texttt{dev\_rehydrated} predictions evaluated against \texttt{dev\_public} gold markers. TP, Pred, and Gold denote true positives, total predicted spans, and total gold spans.}
\label{tab:per_role_extraction}
\end{table}





The RB-DET-AG series applies per-sample augmentation after splitting the training corpus ($N{=}2{,}929$ yes/no samples) from the held-out evaluation set. We evaluate two configurations that differ in the number of augmented variants generated per original sample and in the augmentation methods applied:

\begin{itemize}
    \item \textbf{RB-DET-AG2$\times$} generates \emph{one} augmented variant
          per original sample (target $2\times$ expansion; actual corpus size:
          5,858 samples).  Because the augmentation loop assigns the
          \emph{first} slot (index 0, even) to contextual synonym substitution
          and the \emph{second} slot (index 1, odd) to back-translation,
          RB-DET-AG2$\times$ applies \textbf{contextual synonym substitution
          only}: approximately 5\% of tokens per sentence are replaced with
          contextually plausible alternatives predicted by \texttt{roberta-base}
          via \texttt{nlpaug}~\citep{ma2019nlpaug}.  Back-translation serves only as
          a fallback in this configuration (invoked if the BERT augmenter is
          unavailable).

    \item \textbf{RB-DET-AG3$\times$} generates \emph{two} augmented variants
          per original sample (target $3\times$ expansion; actual corpus size:
          8,787 samples).  The two slots alternate methods, yielding
          approximately equal numbers of each type: (1)~contextual synonym
          substitution (same \texttt{roberta-base} configuration as above) and
          (2)~\textbf{back-translation} via a German (DE) pivot using
          MarianMT~\citep{junczys-dowmunt-etal-2018-marian}: \texttt{Helsinki-NLP/opus-mt-en-de}
          (EN$\to$DE) followed by \texttt{opus-mt-de-en} (DE$\to$EN), decoded
          with the model's default beam-search (\texttt{num\_beams=4}).
\end{itemize}

In both configurations, an augmented sample is accepted only if the output
string differs from the source and is non-empty, providing a minimal
surface-form quality gate.  The two configurations also differ in their
fine-tuning setup: RB-DET-AG2$\times$ unfreezes the top 12 of 24 transformer
layers (lr = $5{\times}10^{-5}$, no weight decay, 20 epochs), while
RB-DET-AG3$\times$ unfreezes the top 18 of 24 layers (lr = $2{\times}10^{-6}$,
weight decay = 0.01, 25 epochs), with embeddings frozen in both cases and
classification heads always trainable.


\subsection{Failure Analysis of RB-DET-AG3$\times$}

As shown in Table~\ref{tab:transformer_results}, neither augmented configuration
consistently outperforms non-augmented contextual baselines.  For
RB-DET-AG2$\times$, which is based solely on BERT-based synonym substitution, the
augmented variants are semantically close to the originals but may smooth out the
idiosyncratic lexical patterns that carry the conspiracy signal.  For
RB-DET-AG3$\times$, where half of all augmented samples are produced by
back-translation, three additional failure modes are observed
(Table~\ref{tab:bt-samples},row~2), and \textit{``gun Mason compass''} becomes \textit{``Cannon Mason Compass''} (row~10), altering semantically critical terms that carry conspiratorial signal.

\begin{itemize}
    \item  \textbf{Lexical hallucination.}  Colloquial or domain-specific tokens are         occasionally replaced by non-existent or semantically distinct words: \textit{``rants''} $\to$ \textit{``ranzes''} (row~4); proper nouns such as \textit{``moshavim''} $\to$ \textit{``Moshabim''} (row~3).  These substitutions introduce vocabulary not present in the original corpus distribution.
    
    \item \textbf{Syntactic restructuring;} Round-trip translation sometimes inverts argument structure, producing ungrammatical or meaning-reversing outputs.  For instance, \textit{``Is there any method of persuasion \ldots\ that you think can penetrate a community that far off the deep end?''} It is rendered as \textit{``Is there a method \ldots\ that you think that a community that can penetrate far away from the deep end?''} (row~7), altering the subject--object relationship entirely.

\end{itemize}

Taken together, these errors introduce \textit{label-preserving but
semantics-altering} samples into training.  Because conspiracy-relevant
linguistic markers (specific phrasings, rhetorical devices, and named entities)
are part of what the model learns to detect, their systematic distortion by the
translation pipeline is likely to attenuate the classifier's sensitivity to
genuine conspiratorial language.  The absence of a semantic consistency filter
(e.g., a minimum SBERT cosine similarity threshold between source and
back-translated text) amplifies this effect at higher augmentation multipliers.
These findings reinforce our adoption of \textbf{RB-DET-LoRA}, which achieves
stronger performance by adapting the model's existing representational capacity
rather than expanding the training set with synthetic text.


\begin{comment}
\begin{table}[t]
\centering
\small
\begin{tabular}{l l r r}
\toprule
\textbf{TreeSHAP} & \textbf{t-test} & \textbf{$t$-statistic} & \textbf{$p$-value} \\
\midrule
adj\_ratio & stopword\_ratio & +3.87 & 0.0001  \\
noun\_ratio & n\_advs\_x       & +3.81 & 0.0001  \\
num\_ratio  & qmarks          & +3.11 & 0.0019  \\
adp\_ratio  & commas          & -3.00 & 0.0027  \\
adv\_ratio  & n\_nouns\_x     & -2.82 & 0.0049  \\
\bottomrule
\end{tabular}
\caption{Comparison of top-ranked features from TreeSHAP (by impact) and univariate tests (by significance).}
\label{tab:treeshap_vs_ttest}
\end{table}
\end{comment}
