Abstract
We investigate parsing replicability across 7 languages (and 8 treebanks), showing that choices concerning the use of grammatical functions in parsing or evaluation, the influence of the rare word threshold, as well as choices in test sentences and evaluation script options have considerable and often unexpected effects on parsing accuracies. All of those choices need to be carefully documented if we want to ensure replicability.- Anthology ID:
- R17-1026
- Volume:
- Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017
- Month:
- September
- Year:
- 2017
- Address:
- Varna, Bulgaria
- Editors:
- Ruslan Mitkov, Galia Angelova
- Venue:
- RANLP
- SIG:
- Publisher:
- INCOMA Ltd.
- Note:
- Pages:
- 185–194
- Language:
- URL:
- https://doi.org/10.26615/978-954-452-049-6_026
- DOI:
- 10.26615/978-954-452-049-6_026
- Cite (ACL):
- Daniel Dakota and Sandra Kübler. 2017. Towards Replicability in Parsing. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 185–194, Varna, Bulgaria. INCOMA Ltd..
- Cite (Informal):
- Towards Replicability in Parsing (Dakota & Kübler, RANLP 2017)
- PDF:
- https://doi.org/10.26615/978-954-452-049-6_026