Creating a Corpus of Gestures and Predicting the Audience Response based on Gestures in Speeches of Donald Trump

Verena Ruf, Costanza Navarretta

Abstract
Gestures are an important component of non–verbal communication. This has an increasing potential in human–computer interaction. For example, Navarretta (2017b) uses sequences of speech and pauses together with co–speech gestures produced by Barack Obama in order to predict audience response, such as applause. The aim of this study is to explore the role of speech pauses and gestures alone as predictors of audience reaction without other types of speech information. For this work, we created a corpus of speeches held by Donald Trump before and during his time as president between 2016 and 2019. The data were transcribed with pause information and co–speech gestures were annotated as well as audience responses. Gestures and long silent pauses of the duration of at least 0.5 seconds are the input of computational models to predict audience reaction. The results of this study indicate that especially head movements and facial expressions play an important role and they confirm that gestures can to some extent be used to predict audience reaction independently of speech.
Anthology ID:
2020.lrec-1.136
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
1081–1088
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.136
DOI:
Bibkey:
Cite (ACL):
Verena Ruf and Costanza Navarretta. 2020. Creating a Corpus of Gestures and Predicting the Audience Response based on Gestures in Speeches of Donald Trump. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1081–1088, Marseille, France. European Language Resources Association.
Cite (Informal):
Creating a Corpus of Gestures and Predicting the Audience Response based on Gestures in Speeches of Donald Trump (Ruf & Navarretta, LREC 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-2023-videos/2020.lrec-1.136.pdf
PDF Search