Hierarchically-Refined Label Attention Network for Sequence Labeling

Leyang Cui, Yue Zhang

Abstract
CRF has been used as a powerful model for statistical sequence labeling. For neural sequence labeling, however, BiLSTM-CRF does not always lead to better results compared with BiLSTM-softmax local classification. This can be because the simple Markov label transition model of CRF does not give much information gain over strong neural encoding. For better representing label sequences, we investigate a hierarchically-refined label attention network, which explicitly leverages label embeddings and captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention. Results on POS tagging, NER and CCG supertagging show that the proposed model not only improves the overall tagging accuracy with similar number of parameters, but also significantly speeds up the training and testing compared to BiLSTM-CRF.
Anthology ID:
D19-1422
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
4115–4128
Language:
URL:
https://aclanthology.org/D19-1422
DOI:
10.18653/v1/D19-1422
Bibkey:
Cite (ACL):
Leyang Cui and Yue Zhang. 2019. Hierarchically-Refined Label Attention Network for Sequence Labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4115–4128, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Hierarchically-Refined Label Attention Network for Sequence Labeling (Cui & Zhang, EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/D19-1422.pdf
Attachment:
 D19-1422.Attachment.rar
Code
 Nealcly/LAN +  additional community code
Data
CCGbank, OntoNotes 5.0, Penn Treebank, Universal Dependencies
PDF Search Code Attachment