Weakly- and Semi-supervised Evidence Extraction
Danish Pruthi, Bhuwan Dhingra, Graham Neubig, Zachary C. Lipton
Abstract
For many prediction tasks, stakeholders desire not only predictions but also supporting evidence that a human can use to verify its correctness. However, in practice, evidence annotations may only be available for a minority of training examples (if available at all). In this paper, we propose new methods to combine few evidence annotations (strong semi-supervision) with abundant document-level labels (weak supervision) for the task of evidence extraction. Evaluating on two classification tasks that feature evidence annotations, we find that our methods outperform baselines adapted from the interpretability literature to our task. Our approach yields gains with as few as hundred evidence annotations.- Anthology ID:
- 2020.findings-emnlp.353
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Trevor Cohn, Yulan He, Yang Liu
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3965–3970
- Language:
- URL:
- https://aclanthology.org/2020.findings-emnlp.353
- DOI:
- 10.18653/v1/2020.findings-emnlp.353
- Cite (ACL):
- Danish Pruthi, Bhuwan Dhingra, Graham Neubig, and Zachary C. Lipton. 2020. Weakly- and Semi-supervised Evidence Extraction. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3965–3970, Online. Association for Computational Linguistics.
- Cite (Informal):
- Weakly- and Semi-supervised Evidence Extraction (Pruthi et al., Findings 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/2020.findings-emnlp.353.pdf
- Code
- danishpruthi/evidence-extraction
- Data
- IMDb Movie Reviews