Abstract
Since obtaining a perfect training dataset (i.e., a dataset which is considerably large, unbiased, and well-representative of unseen cases) is hardly possible, many real-world text classifiers are trained on the available, yet imperfect, datasets. These classifiers are thus likely to have undesirable properties. For instance, they may have biases against some sub-populations or may not work effectively in the wild due to overfitting. In this paper, we propose FIND – a framework which enables humans to debug deep learning text classifiers by disabling irrelevant hidden features. Experiments show that by using FIND, humans can improve CNN text classifiers which were trained under different types of imperfect datasets (including datasets with biases and datasets with dissimilar train-test distributions).- Anthology ID:
- 2020.emnlp-main.24
- Volume:
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 332–348
- Language:
- URL:
- https://aclanthology.org/2020.emnlp-main.24
- DOI:
- 10.18653/v1/2020.emnlp-main.24
- Cite (ACL):
- Piyawat Lertvittayakumjorn, Lucia Specia, and Francesca Toni. 2020. FIND: Human-in-the-Loop Debugging Deep Text Classifiers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 332–348, Online. Association for Computational Linguistics.
- Cite (Informal):
- FIND: Human-in-the-Loop Debugging Deep Text Classifiers (Lertvittayakumjorn et al., EMNLP 2020)
- PDF:
- https://preview.aclanthology.org/starsem-semeval-split/2020.emnlp-main.24.pdf
- Code
- plkumjorn/FIND