Abstract
This paper presents our submission to task 8 (memotion analysis) of the SemEval 2020 competition. We explain the algorithms that were used to learn our models along with the process of tuning the algorithms and selecting the best model. Since meme analysis is a challenging task with two distinct modalities, we studied the impact of different multimodal representation strategies. The results of several approaches to dealing with multimodal data are therefore discussed in the paper. We found that alignment-based strategies did not perform well on memes. Our quantitative results also showed that images and text were uncorrelated. Fusion-based strategies did not show significant improvements and using one modality only (text or image) tends to lead to better results when applied with the predictive models that we used in our research.- Anthology ID:
- 2020.semeval-1.102
- Volume:
- Proceedings of the Fourteenth Workshop on Semantic Evaluation
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona (online)
- Editors:
- Aurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- International Committee for Computational Linguistics
- Note:
- Pages:
- 804–816
- Language:
- URL:
- https://aclanthology.org/2020.semeval-1.102
- DOI:
- 10.18653/v1/2020.semeval-1.102
- Award:
- Best Paper Honorable Mention
- Cite (ACL):
- Lisa Bonheme and Marek Grzes. 2020. SESAM at SemEval-2020 Task 8: Investigating the Relationship between Image and Text in Sentiment Analysis of Memes. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 804–816, Barcelona (online). International Committee for Computational Linguistics.
- Cite (Informal):
- SESAM at SemEval-2020 Task 8: Investigating the Relationship between Image and Text in Sentiment Analysis of Memes (Bonheme & Grzes, SemEval 2020)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2020.semeval-1.102.pdf