Sebastian Lapuschkin
2026
From Weights to Activations: Is Steering the Next Frontier of Adaptation?
Simon Ostermann | Daniil Gurgurov | Tanja Baeumel | Michael A. Hedderich | Sebastian Lapuschkin | Wojciech Samek | Vera Schmitt
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Simon Ostermann | Daniil Gurgurov | Tanja Baeumel | Michael A. Hedderich | Sebastian Lapuschkin | Wojciech Samek | Vera Schmitt
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Post-training adaptation of large language models is commonly achieved through parameter updates or input based methods such as fine-tuning, parameter-efficient adaptation, and prompting. In parallel, a growing body of work modifies internal activations at inference time to influence model behavior, an approach known as *steering*. Despite increasing use, steering is rarely analyzed within the same conceptual framework as established adaptation methods.In this work, we argue that steering should be regarded as a form of model adaptation. We introduce a set of functional criteria for adaptation methods and use them to compare steering approaches with classical alternatives. This analysis positions steering as a distinct adaptation paradigm based on targeted interventions in activation space, enabling local and reversible behavioral change without parameter updates. The resulting framing clarifies how steering relates to existing methods, motivating a unified taxonomy for model adaptation.
2025
FADE: Why Bad Descriptions Happen to Good Features
Bruno Puri | Aakriti Jain | Elena Golimblevskaia | Patrick Kahardipraja | Thomas Wiegand | Wojciech Samek | Sebastian Lapuschkin
Findings of the Association for Computational Linguistics: ACL 2025
Bruno Puri | Aakriti Jain | Elena Golimblevskaia | Patrick Kahardipraja | Thomas Wiegand | Wojciech Samek | Sebastian Lapuschkin
Findings of the Association for Computational Linguistics: ACL 2025
Recent advances in mechanistic interpretability have highlighted the potential of automating interpretability pipelines in analyzing the latent representations within LLMs. While this may enhance our understanding of internal mechanisms, the field lacks standardized evaluation methods for assessing the validity of discovered features. We attempt to bridge this gap by introducing **FADE**: Feature Alignment to Description Evaluation, a scalable model-agnostic framework for automatically evaluating feature-to-description alignment. **FADE** evaluates alignment across four key metrics – *Clarity, Responsiveness, Purity, and Faithfulness* – and systematically quantifies the causes of the misalignment between features and their descriptions. We apply **FADE** to analyze existing open-source feature descriptions and assess key components of automated interpretability pipelines, aiming to enhance the quality of descriptions. Our findings highlight fundamental challenges in generating feature descriptions, particularly for SAEs compared to MLP neurons, providing insights into the limitations and future directions of automated interpretability. We release **FADE** as an open-source package at: [github.com/brunibrun/FADE](https://github.com/brunibrun/FADE).