Weak Ensemble Learning from Multiple Annotators for Subjective Text Classification

Ziyi Huang, N. R. Abeynayake, Xia Cui


Abstract
With the rise of online platforms, moderating harmful or offensive user-generated content has become increasingly critical. As manual moderation is infeasible at scale, machine learning models are widely used to support this process. However, subjective tasks, such as offensive language detection, often suffer from annotator disagreement, resulting in noisy supervision that hinders training and evaluation. We propose Weak Ensemble Learning (WEL), a novel framework that explicitly models annotator disagreement by constructing and aggregating weak predictors derived from diverse annotator perspectives. WEL enables robust learning from subjective and inconsistent labels without requiring annotator metadata. Experiments on four benchmark datasets show that WEL outperforms strong baselines across multiple metrics, demonstrating its effectiveness and flexibility across domains and annotation conditions.
Anthology ID:
2025.nlperspectives-1.8
Volume:
Proceedings of the The 4th Workshop on Perspectivist Approaches to NLP
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Gavin Abercrombie, Valerio Basile, Simona Frenda, Sara Tonelli, Shiran Dudy
Venues:
NLPerspectives | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
87–99
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.nlperspectives-1.8/
DOI:
Bibkey:
Cite (ACL):
Ziyi Huang, N. R. Abeynayake, and Xia Cui. 2025. Weak Ensemble Learning from Multiple Annotators for Subjective Text Classification. In Proceedings of the The 4th Workshop on Perspectivist Approaches to NLP, pages 87–99, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Weak Ensemble Learning from Multiple Annotators for Subjective Text Classification (Huang et al., NLPerspectives 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.nlperspectives-1.8.pdf