Abstract
Multimodal Neural Machine Translation is focusing on using visual information to translate sentences in the source language into the target language. The main idea is to utilise information from visual modalities to promote the output quality of the text-based translation model. Although the recent multimodal strategies extract the most relevant visual information in images, the effectiveness of using visual information on translation quality changes based on the text dataset. Due to this, this work studies the impact of leveraging visual information in multimodal translation models of ambiguous sentences. Our experiments analyse the Multi30k evaluation dataset and calculate ambiguity scores of sentences based on the WordNet hierarchical structure. To calculate the ambiguity of a sentence, we extract the ambiguity scores for all nouns based on the number of senses in WordNet. The main goal is to find in which sentences, visual content can improve the text-based translation model. We report the correlation between the ambiguity scores and translation quality extracted for all sentences in the English-German dataset.- Anthology ID:
- 2022.naacl-srw.12
- Volume:
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop
- Month:
- July
- Year:
- 2022
- Address:
- Hybrid: Seattle, Washington + Online
- Editors:
- Daphne Ippolito, Liunian Harold Li, Maria Leonor Pacheco, Danqi Chen, Nianwen Xue
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 89–95
- Language:
- URL:
- https://aclanthology.org/2022.naacl-srw.12
- DOI:
- 10.18653/v1/2022.naacl-srw.12
- Cite (ACL):
- Ali Hatami, Paul Buitelaar, and Mihael Arcan. 2022. Analysing the Correlation between Lexical Ambiguity and Translation Quality in a Multimodal Setting using WordNet. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, pages 89–95, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
- Cite (Informal):
- Analysing the Correlation between Lexical Ambiguity and Translation Quality in a Multimodal Setting using WordNet (Hatami et al., NAACL 2022)
- PDF:
- https://preview.aclanthology.org/landing_page/2022.naacl-srw.12.pdf