Susan McKeever


2023

pdf
Unimodal Intermediate Training for Multimodal Meme Sentiment Classification
Muzhaffar Hazman | Susan McKeever | Josephine Griffith
Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing

Internet Memes remain a challenging form of user-generated content for automated sentiment classification. The availability of labelled memes is a barrier to developing sentiment classifiers of multimodal memes. To address the shortage of labelled memes, we propose to supplement the training of a multimodal meme classifier with unimodal (image-only and text-only) data. In this work, we present a novel variant of supervised intermediate training that uses relatively abundant sentiment-labelled unimodal data. Our results show a statistically significant performance improvement from the incorporation of unimodal text data. Furthermore, we show that the training set of labelled memes can be reduced by 40% without reducing the performance of the downstream model.