Dynamic Language Models for Streaming Text
Dani Yogatama, Chong Wang, Bryan R. Routledge, Noah A. Smith, Eric P. Xing
Abstract
We present a probabilistic language model that captures temporal dynamics and conditions on arbitrary non-linguistic context features. These context features serve as important indicators of language changes that are otherwise difficult to capture using text data by itself. We learn our model in an efficient online fashion that is scalable for large, streaming data. With five streaming datasets from two different genres—economics news articles and social media—we evaluate our model on the task of sequential language modeling. Our model consistently outperforms competing models.- Anthology ID:
- Q14-1015
- Volume:
- Transactions of the Association for Computational Linguistics, Volume 2
- Month:
- Year:
- 2014
- Address:
- Cambridge, MA
- Editors:
- Dekang Lin, Michael Collins, Lillian Lee
- Venue:
- TACL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 181–192
- Language:
- URL:
- https://aclanthology.org/Q14-1015
- DOI:
- 10.1162/tacl_a_00175
- Cite (ACL):
- Dani Yogatama, Chong Wang, Bryan R. Routledge, Noah A. Smith, and Eric P. Xing. 2014. Dynamic Language Models for Streaming Text. Transactions of the Association for Computational Linguistics, 2:181–192.
- Cite (Informal):
- Dynamic Language Models for Streaming Text (Yogatama et al., TACL 2014)
- PDF:
- https://preview.aclanthology.org/naacl24-info/Q14-1015.pdf