@inproceedings{pan-etal-2020-modeling,
    title = "Modeling Intra and Inter-modality Incongruity for Multi-Modal Sarcasm Detection",
    author = "Pan, Hongliang  and
      Lin, Zheng  and
      Fu, Peng  and
      Qi, Yatao  and
      Wang, Weiping",
    editor = "Cohn, Trevor  and
      He, Yulan  and
      Liu, Yang",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2020.findings-emnlp.124/",
    doi = "10.18653/v1/2020.findings-emnlp.124",
    pages = "1383--1392",
    abstract = "Sarcasm is a pervasive phenomenon in today{'}s social media platforms such as Twitter and Reddit. These platforms allow users to create multi-modal messages, including texts, images, and videos. Existing multi-modal sarcasm detection methods either simply concatenate the features from multi modalities or fuse the multi modalities information in a designed manner. However, they ignore the incongruity character in sarcastic utterance, which is often manifested between modalities or within modalities. Inspired by this, we propose a BERT architecture-based model, which concentrates on both intra and inter-modality incongruity for multi-modal sarcasm detection. To be specific, we are inspired by the idea of self-attention mechanism and design inter-modality attention to capturing inter-modality incongruity. In addition, the co-attention mechanism is applied to model the contradiction within the text. The incongruity information is then used for prediction. The experimental results demonstrate that our model achieves state-of-the-art performance on a public multi-modal sarcasm detection dataset."
}