Abstract
Verbal communication is companied by rich non-verbal signals. The usage of gestures, poses, and facial expressions facilitates the information transmission in verbal channel. However, few computational studies have explored the non-verbal channels with finer theoretical lens. We extract gesture representations from monologue video data and train neural sequential models, in order to study the degree to which non-verbal signals can effectively transmit information. We focus on examining whether the gestures demonstrate the similar pattern of entropy rate constancy (ERC) found in words, as predicted by Information Theory. Positive results are shown to support the assumption, which leads to the conclusion that speakers indeed use simple gestures to convey information that enhances verbal communication, and the production of non-verbal information is rationally organized.- Anthology ID:
- 2022.coling-1.12
- Volume:
- Proceedings of the 29th International Conference on Computational Linguistics
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Editors:
- Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, Seung-Hoon Na
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 134–140
- Language:
- URL:
- https://aclanthology.org/2022.coling-1.12
- DOI:
- Cite (ACL):
- Yang Xu, Yang Cheng, and Riya Bhatia. 2022. Gestures Are Used Rationally: Information Theoretic Evidence from Neural Sequential Models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 134–140, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
- Cite (Informal):
- Gestures Are Used Rationally: Information Theoretic Evidence from Neural Sequential Models (Xu et al., COLING 2022)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2022.coling-1.12.pdf