Abstract
We describe work in progress for training a humanoid robot to produce iconic arm and head gestures as part of task-oriented dialogue interaction. This involves the development and use of a multimodal dialog manager for non-experts to quickly ‘program’ the robot through speech and vision. Using this dialog manager, videos of gesture demonstrations are collected. Motor positions are extracted from these videos to specify motor trajectories where collections of motor trajectories are used to produce robot gestures following a Gaussian mixtures approach. Concluding discussion considers how learned representations may be used for gesture recognition by the robot, and how the framework may mature into a system to address language grounding and semantic representation.- Anthology ID:
- 2021.mmsr-1.9
- Volume:
- Proceedings of the 1st Workshop on Multimodal Semantic Representations (MMSR)
- Month:
- June
- Year:
- 2021
- Address:
- Groningen, Netherlands (Online)
- Venue:
- MMSR
- SIG:
- SIGSEM
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 95–101
- Language:
- URL:
- https://aclanthology.org/2021.mmsr-1.9
- DOI:
- Cite (ACL):
- Michael Brady and Han Du. 2021. Teaching Arm and Head Gestures to a Humanoid Robot through Interactive Demonstration and Spoken Instruction. In Proceedings of the 1st Workshop on Multimodal Semantic Representations (MMSR), pages 95–101, Groningen, Netherlands (Online). Association for Computational Linguistics.
- Cite (Informal):
- Teaching Arm and Head Gestures to a Humanoid Robot through Interactive Demonstration and Spoken Instruction (Brady & Du, MMSR 2021)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2021.mmsr-1.9.pdf