A Spoken Dialogue System for Spatial Question Answering in a Physical Blocks World
Georgiy Platonov, Lenhart Schubert, Benjamin Kane, Aaron Gindi
Abstract
A physical blocks world, despite its relative simplicity, requires (in fully interactive form) a rich set of functional capabilities, ranging from vision to natural language understanding. In this work we tackle spatial question answering in a holistic way, using a vision system, speech input and output mediated by an animated avatar, a dialogue system that robustly interprets spatial queries, and a constraint solver that derives answers based on 3-D spatial modeling. The contributions of this work include a semantic parser that maps spatial questions into logical forms consistent with a general approach to meaning representation, a dialogue manager based on a schema representation, and a constraint solver for spatial questions that provides answers in agreement with human perception. These and other components are integrated into a multi-modal human-computer interaction pipeline.- Anthology ID:
- 2020.sigdial-1.16
- Volume:
- Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue
- Month:
- July
- Year:
- 2020
- Address:
- 1st virtual meeting
- Editors:
- Olivier Pietquin, Smaranda Muresan, Vivian Chen, Casey Kennington, David Vandyke, Nina Dethlefs, Koji Inoue, Erik Ekstedt, Stefan Ultes
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 128–131
- Language:
- URL:
- https://aclanthology.org/2020.sigdial-1.16
- DOI:
- 10.18653/v1/2020.sigdial-1.16
- Cite (ACL):
- Georgiy Platonov, Lenhart Schubert, Benjamin Kane, and Aaron Gindi. 2020. A Spoken Dialogue System for Spatial Question Answering in a Physical Blocks World. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 128–131, 1st virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- A Spoken Dialogue System for Spatial Question Answering in a Physical Blocks World (Platonov et al., SIGDIAL 2020)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/2020.sigdial-1.16.pdf
- Data
- CLEVR