Abstract
We carry out a syntactic analysis of two state-of-the-art sentiment analyzers, Google Cloud Natural Language and Stanford CoreNLP, to assess their classification accuracy on sentences with negative polarity items. We were motivated by the absence of studies investigating sentiment analyzer performance on sentences with polarity items, a common construct in human language. Our analysis focuses on two sentential structures: downward entailment and non-monotone quantifiers; and demonstrates weaknesses of Google Natural Language and CoreNLP in capturing polarity item information. We describe the particular syntactic phenomenon that these analyzers fail to understand that any ideal sentiment analyzer must. We also provide a set of 150 test sentences that any ideal sentiment analyzer must be able to understand.- Anthology ID:
- D18-1141
- Volume:
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Month:
- October-November
- Year:
- 2018
- Address:
- Brussels, Belgium
- Editors:
- Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
- Venue:
- EMNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1122–1127
- Language:
- URL:
- https://aclanthology.org/D18-1141
- DOI:
- 10.18653/v1/D18-1141
- Cite (ACL):
- Rohil Verma, Samuel Kim, and David Walter. 2018. Syntactical Analysis of the Weaknesses of Sentiment Analyzers. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1122–1127, Brussels, Belgium. Association for Computational Linguistics.
- Cite (Informal):
- Syntactical Analysis of the Weaknesses of Sentiment Analyzers (Verma et al., EMNLP 2018)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/D18-1141.pdf