Fine-grained Contrastive Learning for Relation Extraction

William Hogan, Jiacheng Li, Jingbo Shang

Abstract
Recent relation extraction (RE) works have shown encouraging improvements by conducting contrastive learning on silver labels generated by distant supervision before fine-tuning on gold labels. Existing methods typically assume all these silver labels are accurate and treat them equally; however, distant supervision is inevitably noisy–some silver labels are more reliable than others. In this paper, we propose fine-grained contrastive learning (FineCL) for RE, which leverages fine-grained information about which silver labels are and are not noisy to improve the quality of learned relationship representations for RE. We first assess the quality of silver labels via a simple and automatic approach we call “learning order denoising,” where we train a language model to learn these relations and record the order of learned training instances. We show that learning order largely corresponds to label accuracy–early-learned silver labels have, on average, more accurate labels than later-learned silver labels. Then, during pre-training, we increase the weights of accurate labels within a novel contrastive learning objective. Experiments on several RE benchmarks show that FineCL makes consistent and significant performance gains over state-of-the-art methods.
Anthology ID:
2022.emnlp-main.71
Volume:
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1083–1095
Language:
URL:
https://aclanthology.org/2022.emnlp-main.71
DOI:
Bibkey:
Cite (ACL):
William Hogan, Jiacheng Li, and Jingbo Shang. 2022. Fine-grained Contrastive Learning for Relation Extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1083–1095, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Fine-grained Contrastive Learning for Relation Extraction (Hogan et al., EMNLP 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.emnlp-main.71.pdf
PDF Search