Abstract
Active learning has been shown to reduce annotation requirements for numerous natural language processing tasks, including semantic role labeling (SRL). SRL involves labeling argument spans for potentially multiple predicates in a sentence, which makes it challenging to aggregate the numerous decisions into a single score for determining new instances to annotate. In this paper, we apply two ways of aggregating scores across multiple predicates in order to choose query sentences with two methods of estimating model certainty: using the neural network’s outputs and using dropout-based Bayesian Active Learning by Disagreement. We compare these methods with three passive baselines — random sentence selection, random whole-document selection, and selecting sentences with the most predicates — and analyse the effect these strategies have on the learning curve with respect to reducing the number of annotated sentences and predicates to achieve high performance.- Anthology ID:
- 2021.iwcs-1.20
- Volume:
- Proceedings of the 14th International Conference on Computational Semantics (IWCS)
- Month:
- June
- Year:
- 2021
- Address:
- Groningen, The Netherlands (online)
- Editors:
- Sina Zarrieß, Johan Bos, Rik van Noord, Lasha Abzianidze
- Venue:
- IWCS
- SIG:
- SIGSEM
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 212–221
- Language:
- URL:
- https://aclanthology.org/2021.iwcs-1.20
- DOI:
- Cite (ACL):
- Skatje Myers and Martha Palmer. 2021. Tuning Deep Active Learning for Semantic Role Labeling. In Proceedings of the 14th International Conference on Computational Semantics (IWCS), pages 212–221, Groningen, The Netherlands (online). Association for Computational Linguistics.
- Cite (Informal):
- Tuning Deep Active Learning for Semantic Role Labeling (Myers & Palmer, IWCS 2021)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2021.iwcs-1.20.pdf