Exploring Universal Sentence Encoders for Zero-shot Text Classification

Souvika Sarkar, Dongji Feng, Shubhra Kanti Karmaker Santu


Abstract
Universal Sentence Encoder (USE) has gained much popularity recently as a general-purpose sentence encoding technique. As the name suggests, USE is designed to be fairly general and has indeed been shown to achieve superior performances for many downstream NLP tasks. In this paper, we present an interesting “negative” result on USE in the context of zero-shot text classification, a challenging task, which has recently gained much attraction. More specifically, we found some interesting cases of zero-shot text classification, where topic based inference outperformed USE-based inference in terms of F1 score. Further investigation revealed that USE struggles to perform well on data-sets with a large number of labels with high semantic overlaps, while topic-based classification works well for the same.
Anthology ID:
2022.aacl-short.18
Volume:
Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
November
Year:
2022
Address:
Online only
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
135–147
Language:
URL:
https://aclanthology.org/2022.aacl-short.18
DOI:
Bibkey:
Cite (ACL):
Souvika Sarkar, Dongji Feng, and Shubhra Kanti Karmaker Santu. 2022. Exploring Universal Sentence Encoders for Zero-shot Text Classification. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 135–147, Online only. Association for Computational Linguistics.
Cite (Informal):
Exploring Universal Sentence Encoders for Zero-shot Text Classification (Sarkar et al., AACL-IJCNLP 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp-22-attachments/2022.aacl-short.18.pdf
Software:
 2022.aacl-short.18.Software.zip
Dataset:
 2022.aacl-short.18.Dataset.zip