Using Captum to Explain Generative Language Models
Vivek Miglani, Aobo Yang, Aram Markosyan, Diego Garcia-Olano, Narine Kokhlikyan
Abstract
Captum is a comprehensive library for model explainability in PyTorch, offering a range of methods from the interpretability literature to enhance users’ understanding of PyTorch models. In this paper, we introduce new features in Captum that are specifically designed to analyze the behavior of generative language models. We provide an overview of the available functionalities and example applications of their potential for understanding learned associations within generative language models.- Anthology ID:
- 2023.nlposs-1.19
- Volume:
- Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Liling Tan, Dmitrijs Milajevs, Geeticka Chauhan, Jeremy Gwinnup, Elijah Rippeth
- Venues:
- NLPOSS | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 165–173
- Language:
- URL:
- https://aclanthology.org/2023.nlposs-1.19
- DOI:
- 10.18653/v1/2023.nlposs-1.19
- Cite (ACL):
- Vivek Miglani, Aobo Yang, Aram Markosyan, Diego Garcia-Olano, and Narine Kokhlikyan. 2023. Using Captum to Explain Generative Language Models. In Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023), pages 165–173, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Using Captum to Explain Generative Language Models (Miglani et al., NLPOSS-WS 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2023.nlposs-1.19.pdf