Abstract
Cross-modality between vision and language is a key component for effective and efficient communication, and human language processing mechanism successfully integrates information from various modalities to extract the intended meaning. However, incomplete linguistic input, i.e. due to a noisy environment, is one of the challenges for a successful communication. In that case, an incompleteness in one channel can be compensated by information from another one. In this paper, by conducting visual-world paradigm, we investigated the dynamics between syntactically possible gap fillers and the visual arrangements in incomplete German sentences and their effect on overall sentence interpretation.- Anthology ID:
- W19-6128
- Volume:
- Proceedings of the 22nd Nordic Conference on Computational Linguistics
- Month:
- September–October
- Year:
- 2019
- Address:
- Turku, Finland
- Editors:
- Mareike Hartmann, Barbara Plank
- Venue:
- NoDaLiDa
- SIG:
- Publisher:
- Linköping University Electronic Press
- Note:
- Pages:
- 272–280
- Language:
- URL:
- https://aclanthology.org/W19-6128
- DOI:
- Cite (ACL):
- Ozge Alacam. 2019. Enhancing Natural Language Understanding through Cross-Modal Interaction: Meaning Recovery from Acoustically Noisy Speech. In Proceedings of the 22nd Nordic Conference on Computational Linguistics, pages 272–280, Turku, Finland. Linköping University Electronic Press.
- Cite (Informal):
- Enhancing Natural Language Understanding through Cross-Modal Interaction: Meaning Recovery from Acoustically Noisy Speech (Alacam, NoDaLiDa 2019)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/W19-6128.pdf