Psycholinguistics Meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering

Claudio Greco, Barbara Plank, Raquel Fernández, Raffaella Bernardi


Abstract
We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we devise a set of linguistically-informed VQA tasks, which differ by the types of questions involved (Wh-questions and polar questions). We test what impact task difficulty has on continual learning, and whether the order in which a child acquires question types facilitates computational models. Our results show that dramatic forgetting is at play and that task difficulty and order matter. Two well-known current continual learning methods mitigate the problem only to a limiting degree.
Anthology ID:
P19-1350
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3601–3605
Language:
URL:
https://aclanthology.org/P19-1350
DOI:
10.18653/v1/P19-1350
Bibkey:
Cite (ACL):
Claudio Greco, Barbara Plank, Raquel Fernández, and Raffaella Bernardi. 2019. Psycholinguistics Meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3601–3605, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Psycholinguistics Meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering (Greco et al., ACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/P19-1350.pdf
Supplementary:
 P19-1350.Supplementary.pdf
Video:
 https://vimeo.com/384787273
Data
CLEVRVisual Question Answering