Deep Learning in Semantic Kernel Spaces
Danilo Croce, Simone Filice, Giuseppe Castellucci, Roberto Basili
Abstract
Kernel methods enable the direct usage of structured representations of textual data during language learning and inference tasks. Expressive kernels, such as Tree Kernels, achieve excellent performance in NLP. On the other side, deep neural networks have been demonstrated effective in automatically learning feature representations during training. However, their input is tensor data, i.e., they can not manage rich structured information. In this paper, we show that expressive kernels and deep neural networks can be combined in a common framework in order to (i) explicitly model structured information and (ii) learn non-linear decision functions. We show that the input layer of a deep architecture can be pre-trained through the application of the Nystrom low-rank approximation of kernel spaces. The resulting “kernelized” neural network achieves state-of-the-art accuracy in three different tasks.- Anthology ID:
- P17-1032
- Volume:
- Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- July
- Year:
- 2017
- Address:
- Vancouver, Canada
- Editors:
- Regina Barzilay, Min-Yen Kan
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 345–354
- Language:
- URL:
- https://aclanthology.org/P17-1032
- DOI:
- 10.18653/v1/P17-1032
- Cite (ACL):
- Danilo Croce, Simone Filice, Giuseppe Castellucci, and Roberto Basili. 2017. Deep Learning in Semantic Kernel Spaces. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 345–354, Vancouver, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Deep Learning in Semantic Kernel Spaces (Croce et al., ACL 2017)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/P17-1032.pdf