
\subsection{Data Description}
The primary dataset for this study is provided by the SemEval 2026 Task 11 organizers \citep{valentino-etal-2026-semeval}, specifically focusing on Subtask 1. This dataset consists of 960 syllogistic arguments, each labeled for both logical validity and content plausibility. The distribution between valid and invalid arguments, as well as plausible and implausible conclusions, is balanced to ensure that a model cannot rely on label frequency to achieve high performance. 

In addition to the official task data, we used an additional dataset from \citet{bertolazzi-etal-2024-systematic} containing 1,280 syllogisms. This dataset is also divided into plausible and implausible instances.

Because we want to create a dataset with equal valid and invalid syllogisms, we have preprocessed this dataset. Hereby we randomly chose 320 believable and 320 unbelievable syllogisms and picked one of the options that makes the syllogism invalid.

By combining these two datasets we got 2240 syllogisms. The test set of 240 syllogisms  ($\approx 11\%$) is from the original SemEval-Task dataset and is shared between all students, so the results can be compared. The rest of the 2000 syllogisms are split into 1600 syllogisms ($\approx 71\%$) for the training set and 400 syllogisms ($\approx 18\%$) for the development set.

Each data instance includes a unique identifier, a natural language syllogism, the plausibility, and the target validity label. An example from the training set is provided below:

\begin{small}
\begin{verbatim}
{
  "id": 
    "50146f21-d265-4e3a-8d93-8165cdbe89a3",
  "syllogism": 
    "All cars are a type of vehicle. 
    No animal is a car. Therefore, 
    no animal can be a vehicle.",
  "validity": false,
  "plausibility": true
}\end{verbatim}
\end{small}



\subsection{Related Work}

Recent research indicates that while LLMs show promise in reasoning tasks, they are highly sensitive to "content effect" or belief bias \cite{bertolazzi-etal-2024-systematic}. 

Comparative studies between humans and LLMs suggest that models often mirror human cognitive biases, struggling significantly when a logically valid argument leads to an unbelievable conclusion \cite{eisape2023systematic}. To mitigate these biases, recent advancements have moved beyond simple prompting. For instance, the k-CAST (kNN-Based Conditional Activation Steering) method demonstrates that intervening in a model's internal activations during inference time can improve formal reasoning accuracy by up to 15\% little to no negative impact on the model's multilingual abilities \cite{valentino2025mitigating}.

Our contribution builds upon these findings by specifically applying activation steering to the problem of content effect bias and compare its results with Supervised Fine-Tuning. While previous work has explored general reasoning improvements, our approach is novel in its focus on separating the internal representations of "validity" during the reasoning process, using targeted steering to force the model into a more abstract and structural processing.