Semi-Supervised Teacher-Student Architecture for Relation Extraction

Fan Luo, Ajay Nagesh, Rebecca Sharp, Mihai Surdeanu

Abstract
Generating a large amount of training data for information extraction (IE) is either costly (if annotations are created manually), or runs the risk of introducing noisy instances (if distant supervision is used). On the other hand, semi-supervised learning (SSL) is a cost-efficient solution to combat lack of training data. In this paper, we adapt Mean Teacher (Tarvainen and Valpola, 2017), a denoising SSL framework to extract semantic relations between pairs of entities. We explore the sweet spot of amount of supervision required for good performance on this binary relation extraction task. Additionally, different syntax representations are incorporated into our models to enhance the learned representation of sentences. We evaluate our approach on the Google-IISc Distant Supervision (GDS) dataset, which removes test data noise present in all previous distance supervision datasets, which makes it a reliable evaluation benchmark (Jat et al., 2017). Our results show that the SSL Mean Teacher approach nears the performance of fully-supervised approaches even with only 10% of the labeled corpus. Further, the syntax-aware model outperforms other syntax-free approaches across all levels of supervision.
Anthology ID:
W19-1505
Volume:
Proceedings of the Third Workshop on Structured Prediction for NLP
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
29–37
Language:
URL:
https://aclanthology.org/W19-1505
DOI:
10.18653/v1/W19-1505
Bibkey:
Cite (ACL):
Fan Luo, Ajay Nagesh, Rebecca Sharp, and Mihai Surdeanu. 2019. Semi-Supervised Teacher-Student Architecture for Relation Extraction. In Proceedings of the Third Workshop on Structured Prediction for NLP, pages 29–37, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Semi-Supervised Teacher-Student Architecture for Relation Extraction (Luo et al., NAACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/paclic-22-ingestion/W19-1505.pdf
PDF Search