Visually Grounded Interpretation of Noun-Noun Compounds in English

Inga Lang, Lonneke Plas, Malvina Nissim, Albert Gatt


Abstract
Noun-noun compounds (NNCs) occur frequently in the English language. Accurate NNC interpretation, i.e. determining the implicit relationship between the constituents of a NNC, is crucial for the advancement of many natural language processing tasks. Until now, computational NNC interpretation has been limited to approaches involving linguistic representations only. However, much research suggests that grounding linguistic representations in vision or other modalities can increase performance on this and other tasks. Our work is a novel comparison of linguistic and visuo-linguistic representations for the task of NNC interpretation. We frame NNC interpretation as a relation classification task, evaluating on a large, relationally-annotated NNC dataset. We combine distributional word vectors with image vectors to investigate how visual information can help improve NNC interpretation systems. We find that adding visual vectors increases classification performance on our dataset in many cases.
Anthology ID:
2022.cmcl-1.3
Volume:
Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Emmanuele Chersoni, Nora Hollenstein, Cassandra Jacobs, Yohei Oseki, Laurent Prévot, Enrico Santus
Venue:
CMCL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
23–35
Language:
URL:
https://aclanthology.org/2022.cmcl-1.3
DOI:
10.18653/v1/2022.cmcl-1.3
Bibkey:
Cite (ACL):
Inga Lang, Lonneke Plas, Malvina Nissim, and Albert Gatt. 2022. Visually Grounded Interpretation of Noun-Noun Compounds in English. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 23–35, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Visually Grounded Interpretation of Noun-Noun Compounds in English (Lang et al., CMCL 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl-24-ws-corrections/2022.cmcl-1.3.pdf
Video:
 https://preview.aclanthology.org/naacl-24-ws-corrections/2022.cmcl-1.3.mp4
Data
ImageNet