\documentclass[11pt]{article}

% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage{acl}

\usepackage{graphicx}
\usepackage{subcaption}
\usepackage{float} % Optional, for precise placement
\usepackage{tcolorbox} % For the nice prompt boxes
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage{amsmath}

% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}

% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

% This is also not strictly necessary, and may be commented out.
% However, it will improve the aesthetics of text in
% the typewriter font.
\usepackage{inconsolata}

%Including images in your LaTeX document requires adding
%additional package(s)
\usepackage{graphicx}
\usepackage{booktabs}

% If the title and author information does not fit in the area allocated, uncomment the following
%
%\setlength\titlebox{<dim>}
%
% and set <dim> to something 5cm or larger.

\title{d'Olle Grieze at SemEval-2026 Task 11: Comparing the Impact of Supervised Fine-Tuning and Activation Steering on Mitigating Content Effect Bias in Syllogistic Reasoning}


\author{Twan Huiskens \\
  \small\texttt{t.n.huiskens@student.rug.nl} \\\And
  Tian Niezing \\
  \small\texttt{t.c.niezing@student.rug.nl} \\ \And
  Koen Snelten\\
  \small\texttt{k.d.snelten@student.rug.nl}
 } 

\raggedbottom
\begin{document}
\maketitle
\begin{abstract}

We investigate the content effect bias in Large Language Models (LLMs) as part of SemEval 2026 Task 11. We compare the impact of supervised fine-tuning using low-rank adaptation against activation steering across several model families, including LLaMA, Gemma and Qwen. Our results show that SFT improves accuracy, with LLaMa 8B reaching 98.75\% accuracy. Activation steering offers limited effectiveness in mitigating the content effect bias. A logit lens analysis further reveals that fine-tuning successfully shifts the model's focus toward logical structure, specifically within the later layers.

\end{abstract}

\section{Introduction}

\input{sections/introduction}

\section{Background}

\input{sections/background}

\section{System Overview}

\input{sections/method}

\section{Results}

\input{sections/results}

\section{Conclusion}

\input{sections/conclusion}



\bibliography{custom}

\newpage

% --------------- APPENDIX --------------------------
\clearpage
\appendix
\onecolumn % Optional: Force single column for the appendix if your paper is two-column

% Reset counters for Appendix (e.g., Figure A1, Table A1)
\setcounter{figure}{0}
\setcounter{table}{0}
\renewcommand{\thefigure}{A\arabic{figure}}
\renewcommand{\thetable}{A\arabic{table}}

\section{System Prompts}
\label{app:prompts}

The performance of an LLM is very dependent on the input prompt. To analyze the impact of the prompt on the content effect, we have experimented with different prompts: 
\begin{itemize}
    \item A short prompt (zero shot) (Appendix \ref{sec:small_system_prompt})
    \item A long prompt (zero shot)
    \item A prompt in which we prompted the model to output only the tag "valid" or "invalid", without punctuation and JSON (zero shot)
    \item A short prompt (fewshot) (Appendix \ref{sec:small_system_fewshot_prompt})
    \item A long prompt (fewshot)
    \item A prompt in which we prompted the model to output only the tag "valid" or "invalid", without punctuation and JSON (fewshot)
\end{itemize}

We included few-shot prompts in our baseline experiments to ensure a fair comparison with activation steering, which requires examples to define its target direction. Additionally, we tested all prompt variations (short and long; zero-shot and few-shot) using a specific formatting step: separating premises with line breaks. We introduced this structure to determine if clearly separating the premises helps the model better analyze the argument's logical structure.

To ensure a fair comparison, we formatted all inputs using the native chat template provided for each instruction-tuned model. While the structural tags (e.g., [INST], <|user|>) will differ, the core prompt content remains consistent across all models. With these prompts, we also hope to find if there is a correlation between the length of a prompt and the content effect bias. We argue that a larger prompt could potentially force the model to use more of its world knowledge since more knowledge is required to understand the prompt. 

\subsection{Small System Prompt}
\label{sec:small_system_prompt}

\begin{tcolorbox}[colback=gray!5!white, colframe=gray!50!black, title=System Prompt (Zero-Shot)]
\ttfamily\small
You are a formal logic processor. Your task is to determine the logical validity of a given syllogism. CRITICAL: Ignore factual accuracy. Validity requires the conclusion to be an inescapable consequence of the premises. Your response MUST be a single, raw JSON object and nothing else. Do not use markdown or explanations.
\newline
The JSON must be: \{"validity": <boolean>\}
\end{tcolorbox}

\subsection{Small System Few-Shot Prompt}
\label{sec:small_system_fewshot_prompt}

\begin{tcolorbox}[colback=gray!5!white, colframe=gray!50!black, title=System Prompt (Few-Shot)]
\ttfamily\small
You are a formal logic processor. Your task is to determine the logical validity of a given syllogism. CRITICAL: Ignore factual accuracy. Validity requires the conclusion to be an inescapable consequence of the premises. Your response MUST be a single, raw JSON object and nothing else. Do not use markdown or explanations.
\newline
The JSON must be: \{"validity": <boolean>\}

\vspace{0.2cm}
Here are some examples:

Syllogism: All millionaires are poor. Some millionaires are happy. 
Therefore, some poor people are happy. 
\newline
Answer: \{"validity": true\}

\vspace{0.1cm}
Syllogism: All mammals are animals. Camels are mammals. 
Therefore, camels are animals.
\newline
Answer: \{"validity": true\}

\vspace{0.1cm}
Syllogism: No humans are immortal. Some organisms are immortal. 
Therefore, some organisms are humans.
\newline
Answer: \{"validity": false\}

\vspace{0.1cm}
Syllogism: Some Yumboes are Tarasques. All Tarasques are Quinotaurs. 
Some Quinotaurs are Centaurs. No Griffin is a Centaur. 
Therefore, no Griffin is a Yumboe.
\newline
Answer: \{"validity": false\}
\end{tcolorbox}

\clearpage


\section{Baseline scores}

% Baseline scores 
\begin{table}[H]
    \centering
    \begin{tabular}{llll}
        \toprule
        \textbf{Model Name} & \textbf{Prompt} & \textbf{Preprocess} & \textbf{Acc (\%)} \\
        \midrule
        Qwen 3 30B & Small System & split & 71.67 \\
        Qwen 3 4B & Small System & split & 66.25\\
        LLaMa 3.2 1B & Long System & split & 54.58\\
        LLaMa 3.2 3B & Small System & split & 68.75\\
        LLaMa 3.1 8B & Small System & split & 65.00\\
        \textbf{Gemma 3 27B} & \textbf{Small System} & \textbf{split} & \textbf{84.17}\\
        Gemma 3 4B & Small System & split & 75.42\\
        Gemma 3 1B & Small System & none & 56.25\\
        Gemini 3 Pro & Small System & split & 77.50\\
        \midrule
        Qwen 3 30B & Small System (Few-shot) & split & 77.50 \\
        Qwen 3 4B & Small System (Few-shot) & split & 73.33\\
        LLaMa 3.2 1B & Small System (Few-shot) & split & 52.92\\
        LLaMa 3.2 3B & Valid/Invalid Tag (Few-shot) & none & 61.67\\
        LLaMa 3.1 8B & Small System (Few-shot) & split & 70.00\\
        \textbf{Gemma 3 27B} & \textbf{Small System (Few-shot)} & \textbf{split} & \textbf{86.67}\\
        Gemma 3 4B & Small System (Few-shot) & split & 75.42\\
        Gemma 3 1B & Small System (Few-shot) & none & 57.92\\
        \bottomrule
    \end{tabular}
    \caption{Comparison of model accuracy (\%) under different system prompts and preprocessing.}
    \label{tab:model_prompt_accuracy}
\end{table}

\section{Steering hyperparameter selection procedure}\label{app:steering_hyper}

We perform the hyperparameter selection in two stages to find the optimal configuration for each steering approach. In the first stage, we will perform a grid search over the number of in-context learning examples ($n_{\text{icl}} \in \{2, 4, 8\}$) and the number of contrastive prompt pairs ($n_{\text{prompts}} \in \{16, 32, 64\}$). The $n_{\text{icl}}$ parameter will determine how many example syllogisms precede each target syllogism in the prompt, while $n_{\text{prompts}}$ will specify the total number of contrastive pairs used to calculate the steering vector. For each configuration, we will compute steering vectors on the training set and evaluate performance on the development set using a fixed steering coefficient ($\alpha = 2.0$).

In the second stage, we take the best-performing configuration from stage one and conduct an $\alpha$-search to optimize the steering coefficient. We will evaluate $\alpha$ values ranging from 1.0 to 10.0 in increments of 0.5 on the development set. The final optimal configuration (including $n_{\text{icl}}$, $n_{\text{prompts}}$, and $\alpha$) will then be evaluated on the test set. This two-stage search procedure will allow us to separately optimize the vector quality (via $n_{\text{icl}}$ and $n_{\text{prompts}}$) and the application strength (via $\alpha$).


% --- SECTION B: SFT ---
\section{Supervised fine tuning}
\label{app:sft}

\input{appendix/sft}

\clearpage

\section{Activation Steering Results per Model and Approach}\label{app:main_steering_results}

\input{tables/as_main_results}

\section{Activation Steering Grid Search Results}
\label{appendix:as_grid_search_results}

\input{tables/as_grid_search_results}

\section{Per-Quadrant Accuracy for Steering Approaches}
\label{appendix:steering_quadrants}

\input{tables/as_quandrants}

\clearpage

\section{Logit Lens Analysis}
\label{app:logit_lens}

\input{sections/ll_appendix}
\input{appendix/detailed_logit_lens}


\clearpage


\end{document}


