Abstract
The last decade saw a surge in digitisation efforts for ancient manuscripts in Sanskrit. Due to various linguistic peculiarities inherent to the language, even the preliminary tasks such as word segmentation are non-trivial in Sanskrit. Elegant models for Word Segmentation in Sanskrit are indispensable for further syntactic and semantic processing of the manuscripts. Current works in word segmentation for Sanskrit, though commendable in their novelty, often have variations in their objective and evaluation criteria. In this work, we set the record straight. We formally define the objectives and the requirements for the word segmentation task. In order to encourage research in the field and to alleviate the time and effort required in pre-processing, we release a dataset of 115,000 sentences for word segmentation. For each sentence in the dataset we include the input character sequence, ground truth segmentation, and additionally lexical and morphological information about all the phonetically possible segments for the given sentence. In this work, we also discuss the linguistic considerations made while generating the candidate space of the possible segments.- Anthology ID:
- W17-2214
- Volume:
- Proceedings of the Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature
- Month:
- August
- Year:
- 2017
- Address:
- Vancouver, Canada
- Editors:
- Beatrice Alex, Stefania Degaetano-Ortlieb, Anna Feldman, Anna Kazantseva, Nils Reiter, Stan Szpakowicz
- Venue:
- LaTeCH
- SIG:
- SIGHUM
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 105–114
- Language:
- URL:
- https://aclanthology.org/W17-2214
- DOI:
- 10.18653/v1/W17-2214
- Cite (ACL):
- Amrith Krishna, Pavan Kumar Satuluri, and Pawan Goyal. 2017. A Dataset for Sanskrit Word Segmentation. In Proceedings of the Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 105–114, Vancouver, Canada. Association for Computational Linguistics.
- Cite (Informal):
- A Dataset for Sanskrit Word Segmentation (Krishna et al., LaTeCH 2017)
- PDF:
- https://preview.aclanthology.org/naacl24-info/W17-2214.pdf