Syllable-based Neural Thai Word Segmentation
Pattarawat Chormai, Ponrawee Prasertsom, Jin Cheevaprawatdomrong, Attapol Rutherford
Abstract
Word segmentation is a challenging pre-processing step for Thai Natural Language Processing due to the lack of explicit word boundaries. The previous systems rely on powerful neural network architecture alone and ignore linguistic substructures of Thai words. We utilize the linguistic observation that Thai strings can be segmented into syllables, which should narrow down the search space for the word boundaries and provide helpful features. Here, we propose a neural Thai Word Segmenter that uses syllable embeddings to capture linguistic constraints and uses dilated CNN filters to capture the environment of each character. Within this goal, we develop the first ML-based Thai orthographical syllable segmenter, which yields syllable embeddings to be used as features by the word segmenter. Our word segmentation system outperforms the previous state-of-the-art system in both speed and accuracy on both in-domain and out-domain datasets.- Anthology ID:
- 2020.coling-main.407
- Volume:
- Proceedings of the 28th International Conference on Computational Linguistics
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona, Spain (Online)
- Editors:
- Donia Scott, Nuria Bel, Chengqing Zong
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 4619–4637
- Language:
- URL:
- https://aclanthology.org/2020.coling-main.407
- DOI:
- 10.18653/v1/2020.coling-main.407
- Cite (ACL):
- Pattarawat Chormai, Ponrawee Prasertsom, Jin Cheevaprawatdomrong, and Attapol Rutherford. 2020. Syllable-based Neural Thai Word Segmentation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4619–4637, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Cite (Informal):
- Syllable-based Neural Thai Word Segmentation (Chormai et al., COLING 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2020.coling-main.407.pdf