Intrinsic and Extrinsic Evaluation of Spatiotemporal Text Representations in Twitter Streams

Lawrence Phillips, Kyle Shaffer, Dustin Arendt, Nathan Hodas, Svitlana Volkova

Abstract
Language in social media is a dynamic system, constantly evolving and adapting, with words and concepts rapidly emerging, disappearing, and changing their meaning. These changes can be estimated using word representations in context, over time and across locations. A number of methods have been proposed to track these spatiotemporal changes but no general method exists to evaluate the quality of these representations. Previous work largely focused on qualitative evaluation, which we improve by proposing a set of visualizations that highlight changes in text representation over both space and time. We demonstrate usefulness of novel spatiotemporal representations to explore and characterize specific aspects of the corpus of tweets collected from European countries over a two-week period centered around the terrorist attacks in Brussels in March 2016. In addition, we quantitatively evaluate spatiotemporal representations by feeding them into a downstream classification task – event type prediction. Thus, our work is the first to provide both intrinsic (qualitative) and extrinsic (quantitative) evaluation of text representations for spatiotemporal trends.
Anthology ID:
W17-2624
Volume:
Proceedings of the 2nd Workshop on Representation Learning for NLP
Month:
August
Year:
2017
Address:
Vancouver, Canada
Editors:
Phil Blunsom, Antoine Bordes, Kyunghyun Cho, Shay Cohen, Chris Dyer, Edward Grefenstette, Karl Moritz Hermann, Laura Rimell, Jason Weston, Scott Yih
Venue:
RepL4NLP
SIG:
SIGREP
Publisher:
Association for Computational Linguistics
Note:
Pages:
201–210
Language:
URL:
https://preview.aclanthology.org/icon-24-ingestion/W17-2624/
DOI:
10.18653/v1/W17-2624
Bibkey:
Cite (ACL):
Lawrence Phillips, Kyle Shaffer, Dustin Arendt, Nathan Hodas, and Svitlana Volkova. 2017. Intrinsic and Extrinsic Evaluation of Spatiotemporal Text Representations in Twitter Streams. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 201–210, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Intrinsic and Extrinsic Evaluation of Spatiotemporal Text Representations in Twitter Streams (Phillips et al., RepL4NLP 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/icon-24-ingestion/W17-2624.pdf
PDF Search Fix data