Abstract
In classification, there are usually some good features that are indicative of class labels. For example, in sentiment classification, words like good and nice are indicative of the positive sentiment and words like bad and terrible are indicative of the negative sentiment. However, there are also many common features (e.g., words) that are not indicative of any specific class (e.g., voice and screen, which are common to both sentiment classes and are not discriminative for classification). Although deep learning has made significant progresses in generating discriminative features through its powerful representation learning, we believe there is still room for improvement. In this paper, we propose a novel angle to further improve this representation learning, i.e., feature projection. This method projects existing features into the orthogonal space of the common features. The resulting projection is thus perpendicular to the common features and more discriminative for classification. We apply this new method to improve CNN, RNN, Transformer, and Bert based text classification and obtain markedly better results.- Anthology ID:
- 2020.acl-main.726
- Volume:
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2020
- Address:
- Online
- Editors:
- Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8161–8171
- Language:
- URL:
- https://aclanthology.org/2020.acl-main.726
- DOI:
- 10.18653/v1/2020.acl-main.726
- Cite (ACL):
- Qi Qin, Wenpeng Hu, and Bing Liu. 2020. Feature Projection for Improved Text Classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8161–8171, Online. Association for Computational Linguistics.
- Cite (Informal):
- Feature Projection for Improved Text Classification (Qin et al., ACL 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2020.acl-main.726.pdf