Abstract
The major shortcomings of using neural networks with situated agents are that in incremental interaction very few learning examples are available and that their visual sensory representations are quite different from image caption datasets. In this work we adapt and evaluate a few-shot learning approach, Matching Networks (Vinyals et al., 2016), to conversational strategies of a robot interacting with a human tutor in order to efficiently learn to categorise objects that are presented to it and also investigate to what degree transfer learning from pre-trained models on images from different contexts can improve its performance. We discuss the implications of such learning on the nature of semantic representations the system has learned.- Anthology ID:
- 2020.pam-1.7
- Volume:
- Proceedings of the Probability and Meaning Conference (PaM 2020)
- Month:
- June
- Year:
- 2020
- Address:
- Gothenburg
- Editors:
- Christine Howes, Stergios Chatzikyriakidis, Adam Ek, Vidya Somashekarappa
- Venue:
- PaM
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 53–61
- Language:
- URL:
- https://aclanthology.org/2020.pam-1.7
- DOI:
- Cite (ACL):
- José Miguel Cano Santín, Simon Dobnik, and Mehdi Ghanimifard. 2020. Fast visual grounding in interaction: bringing few-shot learning with neural networks to an interactive robot. In Proceedings of the Probability and Meaning Conference (PaM 2020), pages 53–61, Gothenburg. Association for Computational Linguistics.
- Cite (Informal):
- Fast visual grounding in interaction: bringing few-shot learning with neural networks to an interactive robot (Cano Santín et al., PaM 2020)
- PDF:
- https://preview.aclanthology.org/corrections-2024-07/2020.pam-1.7.pdf