MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisition

Guangyue Xu, Parisa Kordjamshidi, Joyce Chai

Abstract
Humans have the ability to learn novel compositional concepts by recalling primitive concepts acquired from past experience and generalizing these primitive concepts to novel compositions. Inspired by the above human’s compositional learning procedure, in this paper, we propose MetaReVision, a retrievalenhanced meta-learning model to solve the visually grounded compositional concept learning problem. The proposed MetaReVision consists of a retrieval module and a meta-learning module which are designed to incorporate retrieved primitive concepts as supporting set to meta-train visual-language models for grounded compositional concept recognition. Through meta-learning from episodes constructed by the retriever, MetaReVision learns a generic compositional representation that can be fast updated to recognize novel composi tional concepts. We create CompCOCO and CompFlickr to benchmark the grounded compositional concept learning. Our experimental results show MetaReVision outperforms other competitive baselines and the retrieval module does plays an important role in this compositional learning process.
Anthology ID:
2023.findings-emnlp.818
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
12224–12236
Language:
URL:
https://preview.aclanthology.org/add_missing_videos/2023.findings-emnlp.818/
DOI:
10.18653/v1/2023.findings-emnlp.818
Bibkey:
Cite (ACL):
Guangyue Xu, Parisa Kordjamshidi, and Joyce Chai. 2023. MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisition. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12224–12236, Singapore. Association for Computational Linguistics.
Cite (Informal):
MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisition (Xu et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/add_missing_videos/2023.findings-emnlp.818.pdf
PDF Search Fix data