Topics in the Haystack: Enhancing Topic Quality through Corpus Expansion
Anton Thielmann, Arik Reuter, Quentin Seifert, Elisabeth Bergherr, Benjamin Säfken
Abstract
Extracting and identifying latent topics in large text corpora have gained increasing importance in Natural Language Processing (NLP). Most models, whether probabilistic models similar to Latent Dirichlet Allocation (LDA) or neural topic models, follow the same underlying approach of topic interpretability and topic extraction. We propose a method that incorporates a deeper understanding of both sentence and document themes, and goes beyond simply analyzing word frequencies in the data. Through simple corpus expansion, our model can detect latent topics that may include uncommon words or neologisms, as well as words not present in the documents themselves. Additionally, we propose several new evaluation metrics based on intruder words and similarity measures in the semantic space. We present correlation coefficients with human identification of intruder words and achieve near-human level results at the word-intrusion task. We demonstrate the competitive performance of our method with a large benchmark study, and achieve superior results compared with state-of-the-art topic modeling and document clustering models. The code is available at the following link: https://github.com/AnFreTh/STREAM.- Anthology ID:
- 2024.cl-2.5
- Volume:
- Computational Linguistics, Volume 50, Issue 2 - June 2023
- Month:
- June
- Year:
- 2024
- Address:
- Cambridge, MA
- Venue:
- CL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 619–655
- Language:
- URL:
- https://aclanthology.org/2024.cl-2.5
- DOI:
- 10.1162/coli_a_00506
- Cite (ACL):
- Anton Thielmann, Arik Reuter, Quentin Seifert, Elisabeth Bergherr, and Benjamin Säfken. 2024. Topics in the Haystack: Enhancing Topic Quality through Corpus Expansion. Computational Linguistics, 50(2):619–655.
- Cite (Informal):
- Topics in the Haystack: Enhancing Topic Quality through Corpus Expansion (Thielmann et al., CL 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.cl-2.5.pdf