Abstract
We propose a novel deep neural architecture for dependency parsing, which is built upon a Transformer Encoder (Vaswani et al. 2017) and a Stack Pointer Network (Ma et al. 2018). We first encode each sentence using a Transformer Network and then the dependency graph is generated by a Stack Pointer Network by selecting the head of each word in the sentence through a head selection process. We evaluate our model on Turkish and English treebanks. The results show that our trasformer-based model learns long term dependencies efficiently compared to sequential models such as recurrent neural networks. Our self attended stack pointer network improves UAS score around 6% upon the LSTM based stack pointer (Ma et al. 2018) for Turkish sentences with a length of more than 20 words.- Anthology ID:
- 2020.icon-main.12
- Volume:
- Proceedings of the 17th International Conference on Natural Language Processing (ICON)
- Month:
- December
- Year:
- 2020
- Address:
- Indian Institute of Technology Patna, Patna, India
- Editors:
- Pushpak Bhattacharyya, Dipti Misra Sharma, Rajeev Sangal
- Venue:
- ICON
- SIG:
- Publisher:
- NLP Association of India (NLPAI)
- Note:
- Pages:
- 90–100
- Language:
- URL:
- https://aclanthology.org/2020.icon-main.12
- DOI:
- Cite (ACL):
- Salih Tuc and Burcu Can. 2020. Self Attended Stack-Pointer Networks for Learning Long Term Dependencies. In Proceedings of the 17th International Conference on Natural Language Processing (ICON), pages 90–100, Indian Institute of Technology Patna, Patna, India. NLP Association of India (NLPAI).
- Cite (Informal):
- Self Attended Stack-Pointer Networks for Learning Long Term Dependencies (Tuc & Can, ICON 2020)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2020.icon-main.12.pdf
- Data
- Penn Treebank