They Are Not All Alike: Answering Different Spatial Questions Requires Different Grounding Strategies

Alberto Testoni, Claudio Greco, Tobias Bianchi, Mauricio Mazuecos, Agata Marcante, Luciana Benotti, Raffaella Bernardi


Abstract
In this paper, we study the grounding skills required to answer spatial questions asked by humans while playing the GuessWhat?! game. We propose a classification for spatial questions dividing them into absolute, relational, and group questions. We build a new answerer model based on the LXMERT multimodal transformer and we compare a baseline with and without visual features of the scene. We are interested in studying how the attention mechanisms of LXMERT are used to answer spatial questions since they require putting attention on more than one region simultaneously and spotting the relation holding among them. We show that our proposed model outperforms the baseline by a large extent (9.70% on spatial questions and 6.27% overall). By analyzing LXMERT errors and its attention mechanisms, we find that our classification helps to gain a better understanding of the skills required to answer different spatial questions.
Anthology ID:
2020.splu-1.4
Volume:
Proceedings of the Third International Workshop on Spatial Language Understanding
Month:
November
Year:
2020
Address:
Online
Venue:
SpLU
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
29–38
Language:
URL:
https://aclanthology.org/2020.splu-1.4
DOI:
10.18653/v1/2020.splu-1.4
Bibkey:
Cite (ACL):
Alberto Testoni, Claudio Greco, Tobias Bianchi, Mauricio Mazuecos, Agata Marcante, Luciana Benotti, and Raffaella Bernardi. 2020. They Are Not All Alike: Answering Different Spatial Questions Requires Different Grounding Strategies. In Proceedings of the Third International Workshop on Spatial Language Understanding, pages 29–38, Online. Association for Computational Linguistics.
Cite (Informal):
They Are Not All Alike: Answering Different Spatial Questions Requires Different Grounding Strategies (Testoni et al., SpLU 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2020.splu-1.4.pdf
Video:
 https://slideslive.com/38940076
Data
GuessWhat?!VisDial