Annotation of Messages from Social Media for Influencer Detection

Kevin Deturck, Damien Nouvel, Namrata Patel, Frédérique Segond

Abstract
To develop an influencer detection system, we designed an influence model based on the analysis of conversations in the “Change My View” debate forum. This led us to identify enunciative features (argumentation, emotion expression, view change, ...) related to influence between participants. In this paper, we present the annotation campaign we conducted to build up a reference corpus on these enunciative features. The annotation task was to identify in social media posts the text segments that corresponded to each enunciative feature. The posts to be annotated were extracted from two social media: the “Change My View” debate forum, with discussions on various topics, and Twitter, with posts from users identified as supporters of ISIS (Islamic State of Iraq and Syria). Over a thousand posts have been double or triple annotated throughout five annotation sessions gathering a total of 27 annotators. Some of the sessions involved the same annotators, which allowed us to analyse the evolution of their annotation work. Most of the sessions resulted in a reconciliation phase between the annotators, allowing for discussion and iterative improvement of the guidelines. We measured and analysed inter-annotator agreements over the course of the sessions, which allowed us to validate our iterative approach.
Anthology ID:
2022.law-1.10
Volume:
Proceedings of the 16th Linguistic Annotation Workshop (LAW-XVI) within LREC2022
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Sameer Pradhan, Sandra Kuebler
Venue:
LAW
SIG:
SIGANN
Publisher:
European Language Resources Association
Note:
Pages:
85–90
Language:
URL:
https://aclanthology.org/2022.law-1.10
DOI:
Bibkey:
Cite (ACL):
Kevin Deturck, Damien Nouvel, Namrata Patel, and Frédérique Segond. 2022. Annotation of Messages from Social Media for Influencer Detection. In Proceedings of the 16th Linguistic Annotation Workshop (LAW-XVI) within LREC2022, pages 85–90, Marseille, France. European Language Resources Association.
Cite (Informal):
Annotation of Messages from Social Media for Influencer Detection (Deturck et al., LAW 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2022.law-1.10.pdf
PDF Search