One format to rule them all – The emtsv pipeline for Hungarian
Balázs Indig, Bálint Sass, Eszter Simon, Iván Mittelholcz, Noémi Vadász, Márton Makrai
Abstract
We present a more efficient version of the e-magyar NLP pipeline for Hungarian called emtsv. It integrates Hungarian NLP tools in a framework whose individual modules can be developed or replaced independently and allows new ones to be added. The design also allows convenient investigation and manual correction of the data flow from one module to another. The improvements we publish include effective communication between the modules and support of the use of individual modules both in the chain and standing alone. Our goals are accomplished using extended tsv (tab separated values) files, a simple, uniform, generic and self-documenting input/output format. Our vision is maintaining the system for a long time and making it easier for external developers to fit their own modules into the system, thus sharing existing competencies in the field of processing Hungarian, a mid-resourced language. The source code is available under LGPL 3.0 license at https://github.com/dlt-rilmta/emtsv .- Anthology ID:
- W19-4018
- Volume:
- Proceedings of the 13th Linguistic Annotation Workshop
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Venue:
- LAW
- SIG:
- SIGANN
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 155–165
- Language:
- URL:
- https://aclanthology.org/W19-4018
- DOI:
- 10.18653/v1/W19-4018
- Cite (ACL):
- Balázs Indig, Bálint Sass, Eszter Simon, Iván Mittelholcz, Noémi Vadász, and Márton Makrai. 2019. One format to rule them all – The emtsv pipeline for Hungarian. In Proceedings of the 13th Linguistic Annotation Workshop, pages 155–165, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- One format to rule them all – The emtsv pipeline for Hungarian (Indig et al., LAW 2019)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/W19-4018.pdf
- Code
- dlt-rilmta/emtsv