Abstract
Recurrent neural network (RNN) has achieved remarkable performance in text categorization. RNN can model the entire sequence and capture long-term dependencies, but it does not do well in extracting key patterns. In contrast, convolutional neural network (CNN) is good at extracting local and position-invariant features. In this paper, we present a novel model named disconnected recurrent neural network (DRNN), which incorporates position-invariance into RNN. By limiting the distance of information flow in RNN, the hidden state at each time step is restricted to represent words near the current position. The proposed model makes great improvements over RNN and CNN models and achieves the best performance on several benchmark datasets for text categorization.- Anthology ID:
- P18-1215
- Volume:
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- July
- Year:
- 2018
- Address:
- Melbourne, Australia
- Editors:
- Iryna Gurevych, Yusuke Miyao
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2311–2320
- Language:
- URL:
- https://aclanthology.org/P18-1215
- DOI:
- 10.18653/v1/P18-1215
- Cite (ACL):
- Baoxin Wang. 2018. Disconnected Recurrent Neural Networks for Text Categorization. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2311–2320, Melbourne, Australia. Association for Computational Linguistics.
- Cite (Informal):
- Disconnected Recurrent Neural Networks for Text Categorization (Wang, ACL 2018)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/P18-1215.pdf
- Data
- AG News, DBpedia, Yahoo! Answers, Yelp