@inproceedings{zhang-abdul-mageed-2022-improving,
    title = "Improving Social Meaning Detection with Pragmatic Masking and Surrogate Fine-Tuning",
    author = "Zhang, Chiyu  and
      Abdul-Mageed, Muhammad",
    editor = "Barnes, Jeremy  and
      De Clercq, Orph{\'e}e  and
      Barriere, Valentin  and
      Tafreshi, Shabnam  and
      Alqahtani, Sawsan  and
      Sedoc, Jo{\~a}o  and
      Klinger, Roman  and
      Balahur, Alexandra",
    booktitle = "Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment {\&} Social Media Analysis",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2022.wassa-1.14/",
    doi = "10.18653/v1/2022.wassa-1.14",
    pages = "141--156",
    abstract = "Masked language models (MLMs) are pre-trained with a denoising objective that is in a mismatch with the objective of downstream fine-tuning. We propose pragmatic masking and surrogate fine-tuning as two complementing strategies that exploit social cues to drive pre-trained representations toward a broad set of concepts useful for a wide class of social meaning tasks. We test our models on 15 different Twitter datasets for social meaning detection. Our methods achieve 2.34{\%} $F_1$ over a competitive baseline, while outperforming domain-specific language models pre-trained on large datasets. Our methods also excel in few-shot learning: with only 5{\%} of training data (severely few-shot), our methods enable an impressive 68.54{\%} average $F_1$. The methods are also language agnostic, as we show in a zero-shot setting involving six datasets from three different languages."
}