David Lazer


2021

pdf
(Mis)alignment Between Stance Expressed in Social Media Data and Public Opinion Surveys
Kenneth Joseph | Sarah Shugars | Ryan Gallagher | Jon Green | Alexi Quintana Mathé | Zijian An | David Lazer
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Stance detection, which aims to determine whether an individual is for or against a target concept, promises to uncover public opinion from large streams of social media data. Yet even human annotation of social media content does not always capture “stance” as measured by public opinion polls. We demonstrate this by directly comparing an individual’s self-reported stance to the stance inferred from their social media data. Leveraging a longitudinal public opinion survey with respondent Twitter handles, we conducted this comparison for 1,129 individuals across four salient targets. We find that recall is high for both “Pro’’ and “Anti’’ stance classifications but precision is variable in a number of cases. We identify three factors leading to the disconnect between text and author stance: temporal inconsistencies, differences in constructs, and measurement errors from both survey respondents and annotators. By presenting a framework for assessing the limitations of stance detection models, this work provides important insight into what stance detection truly measures.

2017

pdf
ConStance: Modeling Annotation Contexts to Improve Stance Classification
Kenneth Joseph | Lisa Friedland | William Hobbs | David Lazer | Oren Tsur
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

Manual annotations are a prerequisite for many applications of machine learning. However, weaknesses in the annotation process itself are easy to overlook. In particular, scholars often choose what information to give to annotators without examining these decisions empirically. For subjective tasks such as sentiment analysis, sarcasm, and stance detection, such choices can impact results. Here, for the task of political stance detection on Twitter, we show that providing too little context can result in noisy and uncertain annotations, whereas providing too strong a context may cause it to outweigh other signals. To characterize and reduce these biases, we develop ConStance, a general model for reasoning about annotations across information conditions. Given conflicting labels produced by multiple annotators seeing the same instances with different contexts, ConStance simultaneously estimates gold standard labels and also learns a classifier for new instances. We show that the classifier learned by ConStance outperforms a variety of baselines at predicting political stance, while the model’s interpretable parameters shed light on the effects of each context.

2015

pdf
A Frame of Mind: Using Statistical Models for Detection of Framing and Agenda Setting Campaigns
Oren Tsur | Dan Calacci | David Lazer
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

2014

pdf
As Long as You Name My Name Right: Social Circles and Social Sentiment in the Hollywood Hearings
Oren Tsur | Dan Calacci | David Lazer
Proceedings of the Joint Workshop on Social Dynamics and Personal Attributes in Social Media