Abstract
Many tasks in natural language processing, such as named entity recognition and slot-filling, involve identifying and labeling specific spans of text. In order to leverage common models, these tasks are often recast as sequence labeling tasks. Each token is given a label and these labels are prefixed with special tokens such as B- or I-. After a model assigns labels to each token, these prefixes are used to group the tokens into spans. Properly parsing these annotations is critical for producing fair and comparable metrics; however, despite its importance, there is not an easy-to-use, standardized, programmatically integratable library to help work with span labeling. To remedy this, we introduce our open-source library, iobes. iobes is used for parsing, converting, and processing spans represented as token level decisions.- Anthology ID:
- 2020.nlposs-1.16
- Volume:
- Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Eunjeong L. Park, Masato Hagiwara, Dmitrijs Milajevs, Nelson F. Liu, Geeticka Chauhan, Liling Tan
- Venue:
- NLPOSS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 115–119
- Language:
- URL:
- https://aclanthology.org/2020.nlposs-1.16
- DOI:
- 10.18653/v1/2020.nlposs-1.16
- Cite (ACL):
- Brian Lester. 2020. iobes: Library for Span Level Processing. In Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS), pages 115–119, Online. Association for Computational Linguistics.
- Cite (Informal):
- iobes: Library for Span Level Processing (Lester, NLPOSS 2020)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/2020.nlposs-1.16.pdf
- Code
- blester125/iobes