Readers vs. Writers vs. Texts: Coping with Different Perspectives of Text Understanding in Emotion Annotation

Sven Buechel, Udo Hahn


Abstract
We here examine how different perspectives of understanding written discourse, like the reader’s, the writer’s or the text’s point of view, affect the quality of emotion annotations. We conducted a series of annotation experiments on two corpora, a popular movie review corpus and a genre- and domain-balanced corpus of standard English. We found statistical evidence that the writer’s perspective yields superior annotation quality overall. However, the quality one perspective yields compared to the other(s) seems to depend on the domain the utterance originates from. Our data further suggest that the popular movie review data set suffers from an atypical bimodal distribution which may decrease model performance when used as a training resource.
Anthology ID:
W17-0801
Volume:
Proceedings of the 11th Linguistic Annotation Workshop
Month:
April
Year:
2017
Address:
Valencia, Spain
Editors:
Nathan Schneider, Nianwen Xue
Venue:
LAW
SIG:
SIGANN
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–12
Language:
URL:
https://aclanthology.org/W17-0801
DOI:
10.18653/v1/W17-0801
Bibkey:
Cite (ACL):
Sven Buechel and Udo Hahn. 2017. Readers vs. Writers vs. Texts: Coping with Different Perspectives of Text Understanding in Emotion Annotation. In Proceedings of the 11th Linguistic Annotation Workshop, pages 1–12, Valencia, Spain. Association for Computational Linguistics.
Cite (Informal):
Readers vs. Writers vs. Texts: Coping with Different Perspectives of Text Understanding in Emotion Annotation (Buechel & Hahn, LAW 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-bitext-workshop/W17-0801.pdf
Code
 JULIELab/EmoBank
Data
SST