To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness

Amulya Gupta, Zhu Zhang

[How to correct problems with metadata yourself]


Abstract
With the recent success of Recurrent Neural Networks (RNNs) in Machine Translation (MT), attention mechanisms have become increasingly popular. The purpose of this paper is two-fold; firstly, we propose a novel attention model on Tree Long Short-Term Memory Networks (Tree-LSTMs), a tree-structured generalization of standard LSTM. Secondly, we study the interaction between attention and syntactic structures, by experimenting with three LSTM variants: bidirectional-LSTMs, Constituency Tree-LSTMs, and Dependency Tree-LSTMs. Our models are evaluated on two semantic relatedness tasks: semantic relatedness scoring for sentence pairs (SemEval 2012, Task 6 and SemEval 2014, Task 1) and paraphrase detection for question pairs (Quora, 2017).
Anthology ID:
P18-1197
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2116–2125
Language:
URL:
https://aclanthology.org/P18-1197
DOI:
10.18653/v1/P18-1197
Bibkey:
Cite (ACL):
Amulya Gupta and Zhu Zhang. 2018. To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2116–2125, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness (Gupta & Zhang, ACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/teach-a-man-to-fish/P18-1197.pdf
Presentation:
 P18-1197.Presentation.pdf
Video:
 https://preview.aclanthology.org/teach-a-man-to-fish/P18-1197.mp4
Data
SICK