@inproceedings{wang-etal-2025-leancode,
    title = "{LEANCODE}: Understanding Models Better for Code Simplification of Pre-trained Large Language Models",
    author = "Wang, Yan  and
      Ding, Ling  and
      Nguyen, Tien N  and
      Wang, Shaohua  and
      Zheng, Yanan",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.acl-long.78/",
    pages = "1551--1567",
    ISBN = "979-8-89176-251-0",
    abstract = "Large Language Models for code often entail significant computational complexity, which grows significantly with the length of the input code sequence. We propose $LeanCode$ for code simplification to reduce training and prediction time, leveraging code contexts in utilizing attention scores to represent the tokens' importance. We advocate for the selective removal of tokens based on the average context-aware attention scores rather than average scores across all inputs. $LeanCode$ uses the attention scores of `CLS' tokens within the encoder for classification tasks, such as code search. It also employs the encoder-decoder attention scores to determine token significance for sequence-to-sequence tasks like code summarization. Our evaluation shows $LeanCode${`}s superiority over the SOTAs $DietCode$ and $SlimCode$, with improvements of 60{\%} and 16{\%} for code search, and 29{\%} and 27{\%} for code summarization, respectively."
}