Abstract
Most recent improvements in NLP come from changes to the neural network architectures modeling the text input. Yet, state-of-the-art models often rely on simple approaches to model the label space, e.g. bigram Conditional Random Fields (CRFs) in sequence tagging. More expressive graphical models are rarely used due to their prohibitive computational cost. In this work, we present an approach for efficiently training and decoding hybrids of graphical models and neural networks based on Gibbs sampling. Our approach is the natural adaptation of SampleRank (Wick et al., 2011) to neural models, and is widely applicable to tasks beyond sequence tagging. We apply our approach to named entity recognition and present a neural skip-chain CRF model, for which exact inference is impractical. The skip-chain model improves over a strong baseline on three languages from CoNLL-02/03. We obtain new state-of-the-art results on Dutch.- Anthology ID:
- 2020.emnlp-main.406
- Volume:
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4999–5011
- Language:
- URL:
- https://aclanthology.org/2020.emnlp-main.406
- DOI:
- 10.18653/v1/2020.emnlp-main.406
- Cite (ACL):
- Sida Gao and Matthew R. Gormley. 2020. Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4999–5011, Online. Association for Computational Linguistics.
- Cite (Informal):
- Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (Gao & Gormley, EMNLP 2020)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/2020.emnlp-main.406.pdf
- Code
- GaoSida/Neural-SampleRank
- Data
- CoNLL 2003