Abstract
We present a modular, end-to-end dialogue system for a situated agent to address a multimodal, natural language dialogue task in which the agent learns complex representations of block structure classes through assertions, demonstrations, and questioning. The concept to learn is provided to the user through a set of positive and negative visual examples, from which the user determines the underlying constraints to be provided to the system in natural language. The system in turn asks questions about demonstrated examples and simulates new examples to check its knowledge and verify the user’s description is complete. We find that this task is non-trivial for users and generates natural language that is varied yet understood by our deep language understanding architecture.- Anthology ID:
- W18-5010
- Volume:
- Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue
- Month:
- July
- Year:
- 2018
- Address:
- Melbourne, Australia
- Editors:
- Kazunori Komatani, Diane Litman, Kai Yu, Alex Papangelis, Lawrence Cavedon, Mikio Nakano
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 89–98
- Language:
- URL:
- https://aclanthology.org/W18-5010
- DOI:
- 10.18653/v1/W18-5010
- Cite (ACL):
- Ian Perera, James Allen, Choh Man Teng, and Lucian Galescu. 2018. A Situated Dialogue System for Learning Structural Concepts in Blocks World. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, pages 89–98, Melbourne, Australia. Association for Computational Linguistics.
- Cite (Informal):
- A Situated Dialogue System for Learning Structural Concepts in Blocks World (Perera et al., SIGDIAL 2018)
- PDF:
- https://preview.aclanthology.org/landing_page/W18-5010.pdf