Comprehensive Part-Of-Speech Tag Set and SVM based POS Tagger for Sinhala
Sandareka Fernando, Surangika Ranathunga, Sanath Jayasena, Gihan Dias
Abstract
This paper presents a new comprehensive multi-level Part-Of-Speech tag set and a Support Vector Machine based Part-Of-Speech tagger for the Sinhala language. The currently available tag set for Sinhala has two limitations: the unavailability of tags to represent some word classes and the lack of tags to capture inflection based grammatical variations of words. The new tag set, presented in this paper overcomes both of these limitations. The accuracy of available Sinhala Part-Of-Speech taggers, which are based on Hidden Markov Models, still falls far behind state of the art. Our Support Vector Machine based tagger achieved an overall accuracy of 84.68% with 59.86% accuracy for unknown words and 87.12% for known words, when the test set contains 10% of unknown words.- Anthology ID:
- W16-3718
- Volume:
- Proceedings of the 6th Workshop on South and Southeast Asian Natural Language Processing (WSSANLP2016)
- Month:
- December
- Year:
- 2016
- Address:
- Osaka, Japan
- Venue:
- WSSANLP
- SIG:
- Publisher:
- The COLING 2016 Organizing Committee
- Note:
- Pages:
- 173–182
- Language:
- URL:
- https://aclanthology.org/W16-3718
- DOI:
- Cite (ACL):
- Sandareka Fernando, Surangika Ranathunga, Sanath Jayasena, and Gihan Dias. 2016. Comprehensive Part-Of-Speech Tag Set and SVM based POS Tagger for Sinhala. In Proceedings of the 6th Workshop on South and Southeast Asian Natural Language Processing (WSSANLP2016), pages 173–182, Osaka, Japan. The COLING 2016 Organizing Committee.
- Cite (Informal):
- Comprehensive Part-Of-Speech Tag Set and SVM based POS Tagger for Sinhala (Fernando et al., WSSANLP 2016)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/W16-3718.pdf