Think Visually: Question Answering through Virtual Imagery

Ankit Goyal, Jian Wang, Jia Deng


Abstract
In this paper, we study the problem of geometric reasoning (a form of visual reasoning) in the context of question-answering. We introduce Dynamic Spatial Memory Network (DSMN), a new deep network architecture that specializes in answering questions that admit latent visual representations, and learns to generate and reason over such representations. Further, we propose two synthetic benchmarks, FloorPlanQA and ShapeIntersection, to evaluate the geometric reasoning capability of QA systems. Experimental results validate the effectiveness of our proposed DSMN for visual thinking tasks.
Anthology ID:
P18-1242
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2598–2608
Language:
URL:
https://aclanthology.org/P18-1242
DOI:
10.18653/v1/P18-1242
Bibkey:
Cite (ACL):
Ankit Goyal, Jian Wang, and Jia Deng. 2018. Think Visually: Question Answering through Virtual Imagery. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2598–2608, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Think Visually: Question Answering through Virtual Imagery (Goyal et al., ACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/fix-dup-bibkey/P18-1242.pdf
Note:
 P18-1242.Notes.pdf
Code
 umich-vl/think_visually
Data
Visual Question Answering