Progressive Class Semantic Matching for Semi-supervised Text Classification

Haiming Xu, Lingqiao Liu, Ehsan Abbasnejad


Abstract
Semi-supervised learning is a promising way to reduce the annotation cost for text-classification. Combining with pre-trained language models (PLMs), e.g., BERT, recent semi-supervised learning methods achieved impressive performance. In this work, we further investigate the marriage between semi-supervised learning and a pre-trained language model. Unlike existing approaches that utilize PLMs only for model parameter initialization, we explore the inherent topic matching capability inside PLMs for building a more powerful semi-supervised learning approach. Specifically, we propose a joint semi-supervised learning process that can progressively build a standard K-way classifier and a matching network for the input text and the Class Semantic Representation (CSR). The CSR will be initialized from the given labeled sentences and progressively updated through the training process. By means of extensive experiments, we show that our method can not only bring remarkable improvement to baselines, but also overall be more stable, and achieves state-of-the-art performance in semi-supervised text classification.
Anthology ID:
2022.naacl-main.219
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3003–3013
Language:
URL:
https://aclanthology.org/2022.naacl-main.219
DOI:
10.18653/v1/2022.naacl-main.219
Bibkey:
Cite (ACL):
Haiming Xu, Lingqiao Liu, and Ehsan Abbasnejad. 2022. Progressive Class Semantic Matching for Semi-supervised Text Classification. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3003–3013, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Progressive Class Semantic Matching for Semi-supervised Text Classification (Xu et al., NAACL 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.naacl-main.219.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2022.naacl-main.219.mp4
Code
 heimingx/pcm
Data
AG NewsIMDb Movie ReviewsYahoo! Answers