Abstract
Pre-trained vision-and-language models have achieved impressive results on a variety of tasks, including ones that require complex reasoning beyond object recognition. However, little is known about how they achieve these results or what their limitations are. In this paper, we focus on a particular linguistic capability, namely the understanding of negation. We borrow techniques from the analysis of language models to investigate the ability of pre-trained vision-and-language models to handle negation. We find that these models severely underperform in the presence of negation.- Anthology ID:
- 2021.blackboxnlp-1.27
- Volume:
- Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Editors:
- Jasmijn Bastings, Yonatan Belinkov, Emmanuel Dupoux, Mario Giulianelli, Dieuwke Hupkes, Yuval Pinter, Hassan Sajjad
- Venue:
- BlackboxNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 350–362
- Language:
- URL:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2021.blackboxnlp-1.27/
- DOI:
- 10.18653/v1/2021.blackboxnlp-1.27
- Cite (ACL):
- Radina Dobreva and Frank Keller. 2021. Investigating Negation in Pre-trained Vision-and-language Models. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 350–362, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Investigating Negation in Pre-trained Vision-and-language Models (Dobreva & Keller, BlackboxNLP 2021)
- PDF:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2021.blackboxnlp-1.27.pdf
- Code
- radidd/vision-and-language-negation