@inproceedings{zhou-etal-2022-dd,
    title = "{DD}-{TIG} at Constraint@{ACL}2022: Multimodal Understanding and Reasoning for Role Labeling of Entities in Hateful Memes",
    author = "Zhou, Ziming  and
      Zhao, Han  and
      Dong, Jingjing  and
      Gao, Jun  and
      Liu, Xiaolong",
    editor = "Chakraborty, Tanmoy  and
      Akhtar, Md. Shad  and
      Shu, Kai  and
      Bernard, H. Russell  and
      Liakata, Maria  and
      Nakov, Preslav  and
      Srivastava, Aseem",
    booktitle = "Proceedings of the Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.constraint-1.2",
    doi = "10.18653/v1/2022.constraint-1.2",
    pages = "12--18",
    abstract = "The memes serve as an important tool in online communication, whereas some hateful memes endanger cyberspace by attacking certain people or subjects. Recent studies address hateful memes detection while further understanding of relationships of entities in memes remains unexplored. This paper presents our work at the Constraint@ACL2022 Shared Task: Hero, Villain and Victim: Dissecting harmful memes for semantic role labelling of entities. In particular, we propose our approach utilizing transformer-based multimodal models through a VCR method with data augmentation, continual pretraining, loss re-weighting, and ensemble learning. We describe the models used, the ways of preprocessing and experiments implementation. As a result, our best model achieves the Macro F1-score of 54.707 on the test set of this shared task.",
}