SCALE: A Scalable Language Engineering Toolkit
Joris Pelemans, Lyan Verwimp, Kris Demuynck, Hugo Van hamme, Patrick Wambacq
Abstract
In this paper we present SCALE, a new Python toolkit that contains two extensions to n-gram language models. The first extension is a novel technique to model compound words called Semantic Head Mapping (SHM). The second extension, Bag-of-Words Language Modeling (BagLM), bundles popular models such as Latent Semantic Analysis and Continuous Skip-grams. Both extensions scale to large data and allow the integration into first-pass ASR decoding. The toolkit is open source, includes working examples and can be found on http://github.com/jorispelemans/scale.- Anthology ID:
- L16-1612
- Volume:
- Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
- Month:
- May
- Year:
- 2016
- Address:
- Portorož, Slovenia
- Editors:
- Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association (ELRA)
- Note:
- Pages:
- 3868–3871
- Language:
- URL:
- https://aclanthology.org/L16-1612
- DOI:
- Cite (ACL):
- Joris Pelemans, Lyan Verwimp, Kris Demuynck, Hugo Van hamme, and Patrick Wambacq. 2016. SCALE: A Scalable Language Engineering Toolkit. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 3868–3871, Portorož, Slovenia. European Language Resources Association (ELRA).
- Cite (Informal):
- SCALE: A Scalable Language Engineering Toolkit (Pelemans et al., LREC 2016)
- PDF:
- https://preview.aclanthology.org/ml4al-ingestion/L16-1612.pdf
- Code
- jorispelemans/scale