Learning from Implicit Information in Natural Language Instructions for Robotic Manipulations

Ozan Arkan Can, Pedro Zuidberg Dos Martires, Andreas Persson, Julian Gaal, Amy Loutfi, Luc De Raedt, Deniz Yuret, Alessandro Saffiotti

Abstract
Human-robot interaction often occurs in the form of instructions given from a human to a robot. For a robot to successfully follow instructions, a common representation of the world and objects in it should be shared between humans and the robot so that the instructions can be grounded. Achieving this representation can be done via learning, where both the world representation and the language grounding are learned simultaneously. However, in robotics this can be a difficult task due to the cost and scarcity of data. In this paper, we tackle the problem by separately learning the world representation of the robot and the language grounding. While this approach can address the challenges in getting sufficient data, it may give rise to inconsistencies between both learned components. Therefore, we further propose Bayesian learning to resolve such inconsistencies between the natural language grounding and a robot’s world representation by exploiting spatio-relational information that is implicitly present in instructions given by a human. Moreover, we demonstrate the feasibility of our approach on a scenario involving a robotic arm in the physical world.
Anthology ID:
W19-1604
Volume:
Proceedings of the Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Archna Bhatia, Yonatan Bisk, Parisa Kordjamshidi, Jesse Thomason
Venue:
RoboNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
29–39
Language:
URL:
https://aclanthology.org/W19-1604
DOI:
10.18653/v1/W19-1604
Bibkey:
Cite (ACL):
Ozan Arkan Can, Pedro Zuidberg Dos Martires, Andreas Persson, Julian Gaal, Amy Loutfi, Luc De Raedt, Deniz Yuret, and Alessandro Saffiotti. 2019. Learning from Implicit Information in Natural Language Instructions for Robotic Manipulations. In Proceedings of the Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP), pages 29–39, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Learning from Implicit Information in Natural Language Instructions for Robotic Manipulations (Can et al., RoboNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/W19-1604.pdf
PDF Search