@inproceedings{hazman-etal-2023-unimodal,
    title = "Unimodal Intermediate Training for Multimodal Meme Sentiment Classification",
    author = "Hazman, Muzhaffar  and
      McKeever, Susan  and
      Griffith, Josephine",
    editor = "Mitkov, Ruslan  and
      Angelova, Galia",
    booktitle = "Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing",
    month = sep,
    year = "2023",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2023.ranlp-1.55/",
    pages = "494--506",
    abstract = "Internet Memes remain a challenging form of user-generated content for automated sentiment classification. The availability of labelled memes is a barrier to developing sentiment classifiers of multimodal memes. To address the shortage of labelled memes, we propose to supplement the training of a multimodal meme classifier with unimodal (image-only and text-only) data. In this work, we present a novel variant of supervised intermediate training that uses relatively abundant sentiment-labelled unimodal data. Our results show a statistically significant performance improvement from the incorporation of unimodal text data. Furthermore, we show that the training set of labelled memes can be reduced by 40{\%} without reducing the performance of the downstream model."
}