Abstract
We examine the behaviour of an aspect-based sentiment classifier built by fine-tuning the BERT BASE model on the SemEval 2016 English dataset. In a set of masking experiments, we examine the extent to which the tokens identified as salient by LIME and a gradient-based method are being used by the classifier. We find that both methods are able to produce faithful rationales, with LIME outperforming the gradient-based method. We also identify a set of manually annotated sentiment expressions for this dataset, and carry out more masking experiments with these as human rationales. The enhanced performance of a classifier that only sees the relevant sentiment expressions suggests that they are not being used to their full potential. A comparison of the LIME and gradient rationales with the sentiment expressions reveals only a moderate level of agreement. Some disagreements are related to the fixed length of the rationales and the tendency of the rationales to contain content words related to the aspect itself.- Anthology ID:
- 2023.findings-acl.807
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2023
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 12751–12769
- Language:
- URL:
- https://aclanthology.org/2023.findings-acl.807
- DOI:
- 10.18653/v1/2023.findings-acl.807
- Cite (ACL):
- Joachim Wagner and Jennifer Foster. 2023. Investigating the Saliency of Sentiment Expressions in Aspect-Based Sentiment Analysis. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12751–12769, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Investigating the Saliency of Sentiment Expressions in Aspect-Based Sentiment Analysis (Wagner & Foster, Findings 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.findings-acl.807.pdf