Abstract
Images are more than a collection of objects or attributes — they represent a web of relationships among interconnected objects. Scene Graph has emerged as a new modality as a structured graphical representation of images. Scene Graph encodes objects as nodes connected via pairwise relations as edges. To support question answering on scene graphs, we propose GraphVQA, a language-guided graph neural network framework that translates and executes a natural language question as multiple iterations of message passing among graph nodes. We explore the design space of GraphVQA framework, and discuss the trade-off of different design choices. Our experiments on GQA dataset show that GraphVQA outperforms the state-of-the-art accuracy by a large margin (88.43% vs. 94.78%).- Anthology ID:
- 2021.maiworkshop-1.12
- Volume:
- Proceedings of the Third Workshop on Multimodal Artificial Intelligence
- Month:
- June
- Year:
- 2021
- Address:
- Mexico City, Mexico
- Editors:
- Amir Zadeh, Louis-Philippe Morency, Paul Pu Liang, Candace Ross, Ruslan Salakhutdinov, Soujanya Poria, Erik Cambria, Kelly Shi
- Venue:
- maiworkshop
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 79–86
- Language:
- URL:
- https://aclanthology.org/2021.maiworkshop-1.12
- DOI:
- 10.18653/v1/2021.maiworkshop-1.12
- Cite (ACL):
- Weixin Liang, Yanhao Jiang, and Zixuan Liu. 2021. GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answering. In Proceedings of the Third Workshop on Multimodal Artificial Intelligence, pages 79–86, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answering (Liang et al., maiworkshop 2021)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2021.maiworkshop-1.12.pdf
- Code
- codexxxl/GraphVQA
- Data
- GQA