From Virtual to Real: A Framework for Verbal Interaction with Robots

Eugene Joseph

Abstract
A Natural Language Understanding (NLU) pipeline integrated with a 3D physics-based scene is a flexible way to develop and test language-based human-robot interaction, by virtualizing people, robot hardware and the target 3D environment. Here, interaction means both controlling robots using language and conversing with them about the user’s physical environment and her daily life. Such a virtual development framework was initially developed for the Bot Colony videogame launched on Steam in June 2014, and has been undergoing improvements since. The framework is focused of developing intuitive verbal interaction with various types of robots. Key robot functions (robot vision and object recognition, path planning and obstacle avoidance, task planning and constraints, grabbing and inverse kinematics), the human participants in the interaction, and the impact of gravity and other forces on the environment are all simulated using commercial 3D tools. The framework can be used as a robotics testbed: the results of our simulations can be compared with the output of algorithms in real robots, to validate such algorithms. A novelty of our framework is support for social interaction with robots - enabling robots to converse about people and objects in the user’s environment, as well as learning about human needs and everyday life topics from their owner.
Anthology ID:
W19-1603
Volume:
Proceedings of the Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Archna Bhatia, Yonatan Bisk, Parisa Kordjamshidi, Jesse Thomason
Venue:
RoboNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
18–28
Language:
URL:
https://aclanthology.org/W19-1603
DOI:
10.18653/v1/W19-1603
Bibkey:
Cite (ACL):
Eugene Joseph. 2019. From Virtual to Real: A Framework for Verbal Interaction with Robots. In Proceedings of the Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP), pages 18–28, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
From Virtual to Real: A Framework for Verbal Interaction with Robots (Joseph, RoboNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/autopr/W19-1603.pdf
PDF Search Fix metadata