@inproceedings{monteiro-paes-etal-2025-multi,
    title = "Multi-Level Explanations for Generative Language Models",
    author = "Monteiro Paes, Lucas  and
      Wei, Dennis  and
      Do, Hyo Jin  and
      Strobelt, Hendrik  and
      Luss, Ronny  and
      Dhurandhar, Amit  and
      Nagireddy, Manish  and
      Natesan Ramamurthy, Karthikeyan  and
      Sattigeri, Prasanna  and
      Geyer, Werner  and
      Ghosh, Soumya",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1553/",
    pages = "32291--32317",
    ISBN = "979-8-89176-251-0",
    abstract = "Despite the increasing use of large language models (LLMs) for context-grounded tasks like summarization and question-answering, understanding what makes an LLM produce a certain response is challenging. We propose Multi-Level Explanations for Generative Language Models (MExGen), a technique to provide explanations for context-grounded text generation. MExGen assigns scores to parts of the context to quantify their influence on the model{'}s output. It extends attribution methods like LIME and SHAP to LLMs used in context-grounded tasks where (1) inference cost is high, (2) input text is long, and (3) the output is text. We conduct a systematic evaluation, both automated and human, of perturbation-based attribution methods for summarization and question answering. The results show that our framework can provide more faithful explanations of generated output than available alternatives, including LLM self-explanations. We open-source code for MExGen as part of the ICX360 toolkit: https://github.com/IBM/ICX360."
}