Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discovery

Yimin Deng, Yuxia Wu, Guoshuai Zhao, Li Zhu, Xueming Qian


Abstract
New intent discovery is a crucial capability for task-oriented dialogue systems. Existing methods focus on transferring in-domain (IND) prior knowledge to out-of-domain (OOD) data through pre-training and clustering stages. They either handle the two processes in a pipeline manner, which exhibits a gap between intent representation and clustering process or use typical contrastive clustering that overlooks the potential supervised signals from the whole data. Besides, they often deal with either open intent discovery or OOD settings individually. To this end, we propose a Pseudo-Label enhanced Prototypical Contrastive Learning (PLPCL) model for uniformed intent discovery. We iteratively utilize pseudo-labels to explore potential positive/negative samples for contrastive learning and bridge the gap between representation and clustering. To enable better knowledge transfer, we design a prototype learning method integrating the supervised and pseudo signals from IND and OOD samples. In addition, our method has been proven effective in two different settings of discovering new intents. Experiments on three benchmark datasets and two task settings demonstrate the effectiveness of our approach.
Anthology ID:
2024.findings-emnlp.443
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7549–7562
Language:
URL:
https://preview.aclanthology.org/fix-sig-urls/2024.findings-emnlp.443/
DOI:
10.18653/v1/2024.findings-emnlp.443
Bibkey:
Cite (ACL):
Yimin Deng, Yuxia Wu, Guoshuai Zhao, Li Zhu, and Xueming Qian. 2024. Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discovery. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7549–7562, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discovery (Deng et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/fix-sig-urls/2024.findings-emnlp.443.pdf