Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods
Nils Feldhus, Leonhard Hennig, Maximilian Dustin Nasert, Christopher Ebert, Robert Schwarzenberg, Sebastian Möller
Abstract
Saliency maps can explain a neural model’s predictions by identifying important input features. They are difficult to interpret for laypeople, especially for instances with many features. In order to make them more accessible, we formalize the underexplored task of translating saliency maps into natural language and compare methods that address two key challenges of this approach – what and how to verbalize. In both automatic and human evaluation setups, using token-level attributions from text classification tasks, we compare two novel methods (search-based and instruction-based verbalizations) against conventional feature importance representations (heatmap visualizations and extractive rationales), measuring simulatability, faithfulness, helpfulness and ease of understanding. Instructing GPT-3.5 to generate saliency map verbalizations yields plausible explanations which include associations, abstractive summarization and commonsense reasoning, achieving by far the highest human ratings, but they are not faithfully capturing numeric information and are inconsistent in their interpretation of the task. In comparison, our search-based, model-free verbalization approach efficiently completes templated verbalizations, is faithful by design, but falls short in helpfulness and simulatability. Our results suggest that saliency map verbalization makes feature attribution explanations more comprehensible and less cognitively challenging to humans than conventional representations.- Anthology ID:
- 2023.nlrse-1.4
- Volume:
- Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE)
- Month:
- June
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Bhavana Dalvi Mishra, Greg Durrett, Peter Jansen, Danilo Neves Ribeiro, Jason Wei
- Venue:
- NLRSE
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 30–46
- Language:
- URL:
- https://aclanthology.org/2023.nlrse-1.4
- DOI:
- 10.18653/v1/2023.nlrse-1.4
- Cite (ACL):
- Nils Feldhus, Leonhard Hennig, Maximilian Dustin Nasert, Christopher Ebert, Robert Schwarzenberg, and Sebastian Möller. 2023. Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods. In Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE), pages 30–46, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods (Feldhus et al., NLRSE 2023)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/2023.nlrse-1.4.pdf