Parallel Creation of Gigaword Corpora for Medium Density Languages - an Interim Report

Péter Halácsy, András Kornai, Péter Németh, Dániel Varga


Abstract
For increased speed in developing gigaword language resources for medium resource density languages we integrated several FOSS tools in the HUN* toolkit. While the speed and efficiency of the resulting pipeline has surpassed our expectations, our experience in developing LDC-style resource packages for Uzbek and Kurdish makes clear that neither the data collection nor the subsequent processing stages can be fully automated.
Anthology ID:
L08-1587
Volume:
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)
Month:
May
Year:
2008
Address:
Marrakech, Morocco
Editors:
Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Daniel Tapias
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2008/pdf/858_paper.pdf
DOI:
Bibkey:
Cite (ACL):
Péter Halácsy, András Kornai, Péter Németh, and Dániel Varga. 2008. Parallel Creation of Gigaword Corpora for Medium Density Languages - an Interim Report. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08), Marrakech, Morocco. European Language Resources Association (ELRA).
Cite (Informal):
Parallel Creation of Gigaword Corpora for Medium Density Languages - an Interim Report (Halácsy et al., LREC 2008)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2008/pdf/858_paper.pdf