BNLP: A Text Annotation Platform for Quality Control of LLM-Generated Annotations
Xinhao Zhuang, Qiongyu Tian, Yalin Chen, Tianle Xin, Yongyong Fu, Yunchao Ling, Guoqing Zhang
Abstract
High-quality annotated data is crucial for NLP, yet manual annotation is costly and difficult to scale in low-resource settings. Large Language Models (LLMs) have demonstrated strong zero-shot and few-shot generalization in NLP tasks, but existing annotation tools either lack LLM support or use LLMs only as one-off pre-annotation engines, without incorporating collaboration or quality control, compromising data reliability. We present BNLP, a text annotation platform that embeds LLM-assisted labeling into a quality-aware collaborative workflow. BNLP treats LLM outputs as intermediate, revisable states and integrates multi-role collaboration, iterative review cycles, and consistency analysis to enable continuous quality monitoring while preserving efficiency gains. BNLP also natively supports AI-ready formats such as Excel and JSON, ensuring seamless data flow from manual annotation to model training. Experiments show that BNLP reduces annotation time by 74.3% and improves annotation quality by 11.6% over purely manual annotation in LLM-assisted settings.- Anthology ID:
- 2026.findings-acl.1185
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2026
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 23675–23684
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.findings-acl.1185/
- DOI:
- Cite (ACL):
- Xinhao Zhuang, Qiongyu Tian, Yalin Chen, Tianle Xin, Yongyong Fu, Yunchao Ling, and Guoqing Zhang. 2026. BNLP: A Text Annotation Platform for Quality Control of LLM-Generated Annotations. In Findings of the Association for Computational Linguistics: ACL 2026, pages 23675–23684, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- BNLP: A Text Annotation Platform for Quality Control of LLM-Generated Annotations (Zhuang et al., Findings 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.findings-acl.1185.pdf