Balancing via Generation for Multi-Class Text Classification Improvement
Naama Tepper, Esther Goldbraich, Naama Zwerdling, George Kour, Ateret Anaby Tavor, Boaz Carmeli
Abstract
Data balancing is a known technique for improving the performance of classification tasks. In this work we define a novel balancing-viageneration framework termed BalaGen. BalaGen consists of a flexible balancing policy coupled with a text generation mechanism. Combined, these two techniques can be used to augment a dataset for more balanced distribution. We evaluate BalaGen on three publicly available semantic utterance classification (SUC) datasets. One of these is a new COVID-19 Q&A dataset published here for the first time. Our work demonstrates that optimal balancing policies can significantly improve classifier performance, while augmenting just part of the classes and under-sampling others. Furthermore, capitalizing on the advantages of balancing, we show its usefulness in all relevant BalaGen framework components. We validate the superiority of BalaGen on ten semantic utterance datasets taken from real-life goaloriented dialogue systems. Based on our results we encourage using data balancing prior to training for text classification tasks.- Anthology ID:
- 2020.findings-emnlp.130
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Trevor Cohn, Yulan He, Yang Liu
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1440–1452
- Language:
- URL:
- https://aclanthology.org/2020.findings-emnlp.130
- DOI:
- 10.18653/v1/2020.findings-emnlp.130
- Cite (ACL):
- Naama Tepper, Esther Goldbraich, Naama Zwerdling, George Kour, Ateret Anaby Tavor, and Boaz Carmeli. 2020. Balancing via Generation for Multi-Class Text Classification Improvement. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1440–1452, Online. Association for Computational Linguistics.
- Cite (Informal):
- Balancing via Generation for Multi-Class Text Classification Improvement (Tepper et al., Findings 2020)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2020.findings-emnlp.130.pdf