Exposing propaganda: an analysis of stylistic cues comparing human annotations and machine classification

Géraud Faye, Benjamin Icard, Morgane Casanova, Julien Chanson, François Maine, François Bancilhon, Guillaume Gadek, Guillaume Gravier, Paul Égré


Abstract
This paper investigates the language of propaganda and its stylistic features. It presents the PPN dataset, standing for Propagandist Pseudo-News, a multisource, multilingual, multimodal dataset composed of news articles extracted from websites identified as propaganda sources by expert agencies. A limited sample from this set was randomly mixed with papers from the regular French press, and their URL masked, to conduct an annotation-experiment by humans, using 11 distinct labels. The results show that human annotators were able to reliably discriminate between the two types of press across each of the labels. We use different NLP techniques to identify the cues used by annotators, and to compare them with machine classification: first the analyzer VAGO to detect discourse vagueness and subjectivity, and then four different classifiers, two based on RoBERTa, one CATS using syntax, and one XGBoost combining syntactic and semantic features.
Anthology ID:
2024.unimplicit-1.6
Volume:
Proceedings of the Third Workshop on Understanding Implicit and Underspecified Language
Month:
March
Year:
2024
Address:
Malta
Editors:
Valentina Pyatkin, Daniel Fried, Elias Stengel-Eskin, Alisa Liu, Sandro Pezzelle
Venues:
unimplicit | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
62–72
Language:
URL:
https://aclanthology.org/2024.unimplicit-1.6
DOI:
Bibkey:
Cite (ACL):
Géraud Faye, Benjamin Icard, Morgane Casanova, Julien Chanson, François Maine, François Bancilhon, Guillaume Gadek, Guillaume Gravier, and Paul Égré. 2024. Exposing propaganda: an analysis of stylistic cues comparing human annotations and machine classification. In Proceedings of the Third Workshop on Understanding Implicit and Underspecified Language, pages 62–72, Malta. Association for Computational Linguistics.
Cite (Informal):
Exposing propaganda: an analysis of stylistic cues comparing human annotations and machine classification (Faye et al., unimplicit-WS 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl24-info/2024.unimplicit-1.6.pdf