Abstract
Recent progress in representation and contrastive learning in NLP has not widely considered the class of sociopragmatic meaning (i.e., meaning in interaction within different language communities). To bridge this gap, we propose a novel framework for learning task-agnostic representations transferable to a wide range of sociopragmatic tasks (e.g., emotion, hate speech, humor, sarcasm). Our framework outperforms other contrastive learning frameworks for both in-domain and out-of-domain data, across both the general and few-shot settings. For example, compared to two popular pre-trained language models, our model obtains an improvement of 11.66 average F1 on 16 datasets when fine-tuned on only 20 training samples per dataset. We also show that our framework improves uniformity and preserves the semantic structure of representations. Our code is available at: https://github.com/UBC-NLP/infodcl- Anthology ID:
- 2023.findings-acl.152
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2023
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2405–2439
- Language:
- URL:
- https://aclanthology.org/2023.findings-acl.152
- DOI:
- 10.18653/v1/2023.findings-acl.152
- Cite (ACL):
- Chiyu Zhang, Muhammad Abdul-Mageed, and Ganesh Jawahar. 2023. Contrastive Learning of Sociopragmatic Meaning in Social Media. In Findings of the Association for Computational Linguistics: ACL 2023, pages 2405–2439, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Contrastive Learning of Sociopragmatic Meaning in Social Media (Zhang et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2023.findings-acl.152.pdf