Rather a Nurse than a Physician - Contrastive Explanations under Investigation
Oliver Eberle, Ilias Chalkidis, Laura Cabello, Stephanie Brandl
Abstract
Contrastive explanations, where one decision is explained *in contrast to another*, are supposed to be closer to how humans explain a decision than non-contrastive explanations, where the decision is not necessarily referenced to an alternative. This claim has never been empirically validated. We analyze four English text-classification datasets (SST2, DynaSent, BIOS and DBpedia-Animals). We fine-tune and extract explanations from three different models (RoBERTa, GTP-2, and T5), each in three different sizes and apply three post-hoc explainability methods (LRP, GradientxInput, GradNorm). We furthermore collect and release human rationale annotations for a subset of 100 samples from the BIOS dataset for contrastive and non-contrastive settings. A cross-comparison between model-based rationales and human annotations, both in contrastive and non-contrastive settings, yields a high agreement between the two settings for models as well as for humans. Moreover, model-based explanations computed in both settings align equally well with human rationales. Thus, we empirically find that humans do not necessarily explain in a contrastive manner.- Anthology ID:
- 2023.emnlp-main.427
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 6907–6920
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.427
- DOI:
- 10.18653/v1/2023.emnlp-main.427
- Cite (ACL):
- Oliver Eberle, Ilias Chalkidis, Laura Cabello, and Stephanie Brandl. 2023. Rather a Nurse than a Physician - Contrastive Explanations under Investigation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6907–6920, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Rather a Nurse than a Physician - Contrastive Explanations under Investigation (Eberle et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2023.emnlp-main.427.pdf