Nonlinear Structural Equation Model Guided Gaussian Mixture Hierarchical Topic Modeling

HeGang Chen, Pengbo Mao, Yuyin Lu, Yanghui Rao


Abstract
Hierarchical topic models, which can extract semantically meaningful topics from a textcorpus in an unsupervised manner and automatically organise them into a topic hierarchy, have been widely used to discover the underlying semantic structure of documents. However, the existing models often assume in the prior that the topic hierarchy is a tree structure, ignoring symmetrical dependenciesbetween topics at the same level. Moreover, the sparsity of text data often complicate the analysis. To address these issues, we propose NSEM-GMHTM as a deep topic model, witha Gaussian mixture prior distribution to improve the model’s ability to adapt to sparse data, which explicitly models hierarchical and symmetric relations between topics through the dependency matrices and nonlinear structural equations. Experiments on widely used datasets show that our NSEM-GMHTM generates more coherent topics and a more rational topic structure when compared to state-of-theart baselines. Our code is available at https: //github.com/nbnbhwyy/NSEM-GMHTM.
Anthology ID:
2023.acl-long.578
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10377–10390
Language:
URL:
https://aclanthology.org/2023.acl-long.578
DOI:
10.18653/v1/2023.acl-long.578
Bibkey:
Cite (ACL):
HeGang Chen, Pengbo Mao, Yuyin Lu, and Yanghui Rao. 2023. Nonlinear Structural Equation Model Guided Gaussian Mixture Hierarchical Topic Modeling. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10377–10390, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Nonlinear Structural Equation Model Guided Gaussian Mixture Hierarchical Topic Modeling (Chen et al., ACL 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2023.acl-long.578.pdf
Video:
 https://preview.aclanthology.org/nschneid-patch-4/2023.acl-long.578.mp4