Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers
Brian Lester, Daniel Pressel, Amy Hemmeter, Sagnik Ray Choudhury, Srinivas Bangalore
Abstract
Current state-of-the-art models for named entity recognition (NER) are neural models with a conditional random field (CRF) as the final layer. Entities are represented as per-token labels with a special structure in order to decode them into spans. Current work eschews prior knowledge of how the span encoding scheme works and relies on the CRF learning which transitions are illegal and which are not to facilitate global coherence. We find that by constraining the output to suppress illegal transitions we can train a tagger with a cross-entropy loss twice as fast as a CRF with differences in F1 that are statistically insignificant, effectively eliminating the need for a CRF. We analyze the dynamics of tag co-occurrence to explain when these constraints are most effective and provide open source implementations of our tagger in both PyTorch and TensorFlow.- Anthology ID:
- 2020.findings-emnlp.166
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Trevor Cohn, Yulan He, Yang Liu
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1841–1848
- Language:
- URL:
- https://aclanthology.org/2020.findings-emnlp.166
- DOI:
- 10.18653/v1/2020.findings-emnlp.166
- Cite (ACL):
- Brian Lester, Daniel Pressel, Amy Hemmeter, Sagnik Ray Choudhury, and Srinivas Bangalore. 2020. Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1841–1848, Online. Association for Computational Linguistics.
- Cite (Informal):
- Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers (Lester et al., Findings 2020)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/2020.findings-emnlp.166.pdf
- Code
- blester125/constrained-decoding
- Data
- CoNLL 2003, SNIPS, WNUT 2017