Fairness in classification tasks has traditionally focused on bias removal from neural representations, but recent approaches have shifted towards algorithmic methods that embed fairness into the training process. These methods steer models towards fair performance, preventing potential elimination of valuable information that arises from representation manipulation. Reinforcement Learning (RL), with its ability to learn through interaction and adjust reward functions to encourage desired behaviors, presents a promising approach in this domain. In this paper, we conduct an exploratory evaluation of RL for addressing bias in imbalanced classification by scaling the reward function. We employ the contextual multi-armed bandit framework, adapt three popular RL algorithms, and conduct an extensive empirical evaluation of their relative strengths and limitations. Through this analysis, we contribute meaningful evidence to the ongoing debate between algorithmic and representational fairness approaches.

Reliance on spurious correlations (shortcuts) has been shown to underlie many of the successes of language models. Previous work focused on identifying the input elements that impact prediction. We investigate how shortcuts are actually processed within the model’s decision-making mechanism.We use actor names in movie reviews as controllable shortcuts with known impact on the outcome. We use mechanistic interpretability methods and identify specific attention heads that focus on shortcuts. These heads gear the model towards a label before processing the complete input, effectively making premature decisions that bypass contextual analysis. Based on these findings, we introduce Head-based Token Attribution (HTA), which traces intermediate decisions back to input tokens. We show that HTA is effective in detecting shortcuts in LLMs and enables targeted mitigation by selectively deactivating shortcut-related attention heads.