Abstract
Inconsistencies are part of any manually annotated corpus. Automatically finding these inconsistencies and correcting them (even manually) can increase the quality of the data. Past research has focused mainly on detecting inconsistency in syntactic annotation. This work explores new approaches to detecting inconsistency in semantic annotation. Two ranking methods are presented in this paper: a discrepancy ranking and an entropy ranking. Those methods are then tested and evaluated on multiple corpora annotated with multiword expressions and supersense labels. The results show considerable improvements in detecting inconsistency candidates over a random baseline. Possible applications of methods for inconsistency detection are improving the annotation procedure as well as the guidelines and correcting errors in completed annotations.- Anthology ID:
- L16-1629
- Volume:
- Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
- Month:
- May
- Year:
- 2016
- Address:
- Portorož, Slovenia
- Editors:
- Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association (ELRA)
- Note:
- Pages:
- 3986–3990
- Language:
- URL:
- https://aclanthology.org/L16-1629
- DOI:
- Cite (ACL):
- Nora Hollenstein, Nathan Schneider, and Bonnie Webber. 2016. Inconsistency Detection in Semantic Annotation. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 3986–3990, Portorož, Slovenia. European Language Resources Association (ELRA).
- Cite (Informal):
- Inconsistency Detection in Semantic Annotation (Hollenstein et al., LREC 2016)
- PDF:
- https://preview.aclanthology.org/ml4al-ingestion/L16-1629.pdf
- Code
- norahollenstein/inconsistency-detection