TAAL: Target-Aware Active Learning
Kunal Kotian, Indranil Bhattacharya, Shikhar Gupta, Kaushik Pavani, Naval Bhandari, Sunny Dasgupta
Abstract
Pool-based active learning techniques have had success producing multi-class classifiers that achieve high accuracy with fewer labels com- pared to random labeling. However, in an industrial setting where we often have class-level business targets to achieve (e.g., 95% recall at 95% precision for each class), active learning techniques continue to acquire labels for classes that have already met their targets, thus consuming unnecessary manual annotations. We address this problem by proposing a framework called Target-Aware Active Learning that converts any active learning query strategy into its target-aware variant by leveraging the gap between each class’ current estimated accuracy and its corresponding business target. We show empirically that target-aware variants of state-of-the-art active learning techniques achieve business targets faster on 2 open-source image classification datasets and 2 proprietary product classification datasets.- Anthology ID:
- 2024.ecnlp-1.14
- Volume:
- Proceedings of the Seventh Workshop on e-Commerce and NLP @ LREC-COLING 2024
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Shervin Malmasi, Besnik Fetahu, Nicola Ueffing, Oleg Rokhlenko, Eugene Agichtein, Ido Guy
- Venues:
- ECNLP | WS
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 136–144
- Language:
- URL:
- https://aclanthology.org/2024.ecnlp-1.14
- DOI:
- Cite (ACL):
- Kunal Kotian, Indranil Bhattacharya, Shikhar Gupta, Kaushik Pavani, Naval Bhandari, and Sunny Dasgupta. 2024. TAAL: Target-Aware Active Learning. In Proceedings of the Seventh Workshop on e-Commerce and NLP @ LREC-COLING 2024, pages 136–144, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- TAAL: Target-Aware Active Learning (Kotian et al., ECNLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2024.ecnlp-1.14.pdf