@inproceedings{katz-wolf-2025-reversed,
    title = "Reversed Attention: On The Gradient Descent Of Attention Layers In {GPT}",
    author = "Katz, Shahar  and
      Wolf, Lior",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.naacl-long.52/",
    pages = "1125--1152",
    ISBN = "979-8-89176-189-6",
    abstract = "The success of Transformer-based Language Models (LMs) stems from their attention mechanism. While this mechanism has been extensively studied in explainability research, particularly through the attention values obtained during the forward pass of LMs, the backward pass of attention has been largely overlooked.In this work, we study the mathematics of the backward pass of attention, revealing that it implicitly calculates an attention matrix we refer to as ``Reversed Attention''.We visualized Reversed Attention and examine its properties, demonstrating its ability to elucidate the models' behavior and edit dynamics.In an experimental setup, we showcase the ability of Reversed Attention to directly alter the forward pass of attention, without modifying the model{'}s weights, using a novel method called ``attention patching''.In addition to enhancing the comprehension of how LMs configure attention layers during backpropagation, Reversed Attention maps contribute to a more interpretable backward pass."
}