E-VarM: Enhanced Variational Word Masks to Improve the Interpretability of Text Classification Models
Ling Ge, ChunMing Hu, Guanghui Ma, Junshuang Wu, Junfan Chen, JiHong Liu, Hong Zhang, Wenyi Qin, Richong Zhang
Abstract
Enhancing the interpretability of text classification models can help increase the reliability of these models in real-world applications. Currently, most researchers focus on extracting task-specific words from inputs to improve the interpretability of the model. The competitive approaches exploit the Variational Information Bottleneck (VIB) to improve the performance of word masking at the word embedding layer to obtain task-specific words. However, these approaches ignore the multi-level semantics of the text, which can impair the interpretability of the model, and do not consider the risk of representation overlap caused by the VIB, which can impair the classification performance. In this paper, we propose an enhanced variational word masks approach, named E-VarM, to solve these two issues effectively. The E-VarM combines multi-level semantics from all hidden layers of the model to mask out task-irrelevant words and uses contrastive learning to readjust the distances between representations. Empirical studies on ten benchmark text classification datasets demonstrate that our approach outperforms the SOTA methods in simultaneously improving the interpretability and accuracy of the model.- Anthology ID:
- 2022.coling-1.87
- Volume:
- Proceedings of the 29th International Conference on Computational Linguistics
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Editors:
- Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, Seung-Hoon Na
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 1036–1050
- Language:
- URL:
- https://aclanthology.org/2022.coling-1.87
- DOI:
- Cite (ACL):
- Ling Ge, ChunMing Hu, Guanghui Ma, Junshuang Wu, Junfan Chen, JiHong Liu, Hong Zhang, Wenyi Qin, and Richong Zhang. 2022. E-VarM: Enhanced Variational Word Masks to Improve the Interpretability of Text Classification Models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1036–1050, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
- Cite (Informal):
- E-VarM: Enhanced Variational Word Masks to Improve the Interpretability of Text Classification Models (Ge et al., COLING 2022)
- PDF:
- https://preview.aclanthology.org/corrections-2024-07/2022.coling-1.87.pdf
- Data
- AG News, GLUE, IMDb Movie Reviews, QNLI, SST, SST-2