Abstract
We introduce a class of convolutional neural networks (CNNs) that utilize recurrent neural networks (RNNs) as convolution filters. A convolution filter is typically implemented as a linear affine transformation followed by a non-linear function, which fails to account for language compositionality. As a result, it limits the use of high-order filters that are often warranted for natural language processing tasks. In this work, we model convolution filters with RNNs that naturally capture compositionality and long-term dependencies in language. We show that simple CNN architectures equipped with recurrent neural filters (RNFs) achieve results that are on par with the best published ones on the Stanford Sentiment Treebank and two answer sentence selection datasets.- Anthology ID:
- D18-1109
- Volume:
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Month:
- October-November
- Year:
- 2018
- Address:
- Brussels, Belgium
- Venue:
- EMNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 912–917
- Language:
- URL:
- https://aclanthology.org/D18-1109
- DOI:
- 10.18653/v1/D18-1109
- Cite (ACL):
- Yi Yang. 2018. Convolutional Neural Networks with Recurrent Neural Filters. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 912–917, Brussels, Belgium. Association for Computational Linguistics.
- Cite (Informal):
- Convolutional Neural Networks with Recurrent Neural Filters (Yang, EMNLP 2018)
- PDF:
- https://preview.aclanthology.org/nodalida-main-page/D18-1109.pdf
- Code
- bloomberg/cnn-rnf + additional community code
- Data
- GLUE, SST, WikiQA