Richard Alies
2025
Measuring Label Ambiguity in Subjective Tasks using Predictive Uncertainty Estimation
Richard Alies
|
Elena Merdjanovska
|
Alan Akbik
Proceedings of the 19th Linguistic Annotation Workshop (LAW-XIX-2025)
Human annotations in natural language corpora vary due to differing human perspectives. This is especially prevalent in subjective tasks. In these datasets, certain data samples are more prone to label variation and can be indicated as ambiguous samples.