Abstract
Grounded semantics is typically learnt from utterance-level meaning representations (e.g., successful database retrievals, denoted objects in images, moves in a game). We explore learning word and utterance meanings by continuous observation of the actions of an instruction follower (IF). While an instruction giver (IG) provided a verbal description of a configuration of objects, IF recreated it using a GUI. Aligning these GUI actions to sub-utterance chunks allows a simple maximum entropy model to associate them as chunk meaning better than just providing it with the utterance-final configuration. This shows that semantics useful for incremental (word-by-word) application, as required in natural dialogue, might also be better acquired from incremental settings.- Anthology ID:
- E17-2079
- Volume:
- Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers
- Month:
- April
- Year:
- 2017
- Address:
- Valencia, Spain
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 491–496
- Language:
- URL:
- https://aclanthology.org/E17-2079
- DOI:
- Cite (ACL):
- Ting Han and David Schlangen. 2017. Grounding Language by Continuous Observation of Instruction Following. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 491–496, Valencia, Spain. Association for Computational Linguistics.
- Cite (Informal):
- Grounding Language by Continuous Observation of Instruction Following (Han & Schlangen, EACL 2017)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/E17-2079.pdf