Integration of Knowledge Graph Embedding Into Topic Modeling with Hierarchical Dirichlet Process

Dingcheng Li, Siamak Zamani, Jingyuan Zhang, Ping Li

Abstract
Leveraging domain knowledge is an effective strategy for enhancing the quality of inferred low-dimensional representations of documents by topic models. In this paper, we develop topic modeling with knowledge graph embedding (TMKGE), a Bayesian nonparametric model to employ knowledge graph (KG) embedding in the context of topic modeling, for extracting more coherent topics. Specifically, we build a hierarchical Dirichlet process (HDP) based model to flexibly borrow information from KG to improve the interpretability of topics. An efficient online variational inference method based on a stick-breaking construction of HDP is developed for TMKGE, making TMKGE suitable for large document corpora and KGs. Experiments on three public datasets illustrate the superior performance of TMKGE in terms of topic coherence and document classification accuracy, compared to state-of-the-art topic modeling methods.
Anthology ID:
N19-1099
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Jill Burstein, Christy Doran, Thamar Solorio
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
940–950
Language:
URL:
https://aclanthology.org/N19-1099
DOI:
10.18653/v1/N19-1099
Bibkey:
Cite (ACL):
Dingcheng Li, Siamak Zamani, Jingyuan Zhang, and Ping Li. 2019. Integration of Knowledge Graph Embedding Into Topic Modeling with Hierarchical Dirichlet Process. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 940–950, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Integration of Knowledge Graph Embedding Into Topic Modeling with Hierarchical Dirichlet Process (Li et al., NAACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp22-frontmatter/N19-1099.pdf
PDF Search