BNLP: A Text Annotation Platform for Quality Control of LLM-Generated Annotations

Xinhao Zhuang, Qiongyu Tian, Yalin Chen, Tianle Xin, Yongyong Fu, Yunchao Ling, Guoqing Zhang


Abstract
High-quality annotated data is crucial for NLP, yet manual annotation is costly and difficult to scale in low-resource settings. Large Language Models (LLMs) have demonstrated strong zero-shot and few-shot generalization in NLP tasks, but existing annotation tools either lack LLM support or use LLMs only as one-off pre-annotation engines, without incorporating collaboration or quality control, compromising data reliability. We present BNLP, a text annotation platform that embeds LLM-assisted labeling into a quality-aware collaborative workflow. BNLP treats LLM outputs as intermediate, revisable states and integrates multi-role collaboration, iterative review cycles, and consistency analysis to enable continuous quality monitoring while preserving efficiency gains. BNLP also natively supports AI-ready formats such as Excel and JSON, ensuring seamless data flow from manual annotation to model training. Experiments show that BNLP reduces annotation time by 74.3% and improves annotation quality by 11.6% over purely manual annotation in LLM-assisted settings.
Anthology ID:
2026.findings-acl.1185
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
23675–23684
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.findings-acl.1185/
DOI:
Bibkey:
Cite (ACL):
Xinhao Zhuang, Qiongyu Tian, Yalin Chen, Tianle Xin, Yongyong Fu, Yunchao Ling, and Guoqing Zhang. 2026. BNLP: A Text Annotation Platform for Quality Control of LLM-Generated Annotations. In Findings of the Association for Computational Linguistics: ACL 2026, pages 23675–23684, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
BNLP: A Text Annotation Platform for Quality Control of LLM-Generated Annotations (Zhuang et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.findings-acl.1185.pdf
Checklist:
 2026.findings-acl.1185.checklist.pdf