Enhancing Neural Topic Model with Multi-Level Supervisions from Seed Words

Yang Lin, Xin Gao, Xu Chu, Yasha Wang, Junfeng Zhao, Chao Chen

Abstract
Efforts have been made to apply topic seed words to improve the topic interpretability of topic models. However, due to the semantic diversity of natural language, supervisions from seed words could be ambiguous, making it hard to be incorporated into the current neural topic models. In this paper, we propose SeededNTM, a neural topic model enhanced with supervisions from seed words on both word and document levels. We introduce a context-dependency assumption to alleviate the ambiguities with context document information, and an auto-adaptation mechanism to automatically balance between multi-level information. Moreover, an intra-sample consistency regularizer is proposed to deal with noisy supervisions via encouraging perturbation and semantic consistency. Extensive experiments on multiple datasets show that SeededNTM can derive semantically meaningful topics and outperforms the state-of-the-art seeded topic models in terms of topic quality and classification accuracy.
Anthology ID:
2023.findings-acl.845
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
13361–13377
Language:
URL:
https://aclanthology.org/2023.findings-acl.845
DOI:
10.18653/v1/2023.findings-acl.845
Bibkey:
Cite (ACL):
Yang Lin, Xin Gao, Xu Chu, Yasha Wang, Junfeng Zhao, and Chao Chen. 2023. Enhancing Neural Topic Model with Multi-Level Supervisions from Seed Words. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13361–13377, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Enhancing Neural Topic Model with Multi-Level Supervisions from Seed Words (Lin et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp-22-attachments/2023.findings-acl.845.pdf
PDF Search