Abstract
We present a three-part toolkit for developing morphological analyzers for languages without natural word boundaries. The first part is a C++11/14 lattice-based morphological analysis library that uses a combination of linear and recurrent neural net language models for analysis. The other parts are a tool for exposing problems in the trained model and a partial annotation tool. Our morphological analyzer of Japanese achieves new SOTA on Jumandic-based corpora while being 250 times faster than the previous one. We also perform a small experiment and quantitive analysis and experience of using development tools. All components of the toolkit is open source and available under a permissive Apache 2 License.- Anthology ID:
- D18-2010
- Volume:
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
- Month:
- November
- Year:
- 2018
- Address:
- Brussels, Belgium
- Editors:
- Eduardo Blanco, Wei Lu
- Venue:
- EMNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 54–59
- Language:
- URL:
- https://aclanthology.org/D18-2010
- DOI:
- 10.18653/v1/D18-2010
- Cite (ACL):
- Arseny Tolmachev, Daisuke Kawahara, and Sadao Kurohashi. 2018. Juman++: A Morphological Analysis Toolkit for Scriptio Continua. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 54–59, Brussels, Belgium. Association for Computational Linguistics.
- Cite (Informal):
- Juman++: A Morphological Analysis Toolkit for Scriptio Continua (Tolmachev et al., EMNLP 2018)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/D18-2010.pdf
- Code
- ku-nlp/jumanpp