IIT (BHU) System for Indo-Aryan Language Identification (ILI) at VarDial 2018
Divyanshu Gupta, Gourav Dhakad, Jayprakash Gupta, Anil Kumar Singh
Abstract
Text language Identification is a Natural Language Processing task of identifying and recognizing a given language out of many different languages from a piece of text. This paper describes our submission to the ILI 2018 shared-task, which includes the identification of 5 closely related Indo-Aryan languages. We developed a word-level LSTM(Long Short-term Memory) model, a specific type of Recurrent Neural Network model, for this task. Given a sentence, our model embeds each word of the sentence and convert into its trainable word embedding, feeds them into our LSTM network and finally predict the language. We obtained an F1 macro score of 0.836, ranking 5th in the task.- Anthology ID:
- W18-3921
- Volume:
- Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018)
- Month:
- August
- Year:
- 2018
- Address:
- Santa Fe, New Mexico, USA
- Editors:
- Marcos Zampieri, Preslav Nakov, Nikola Ljubešić, Jörg Tiedemann, Shervin Malmasi, Ahmed Ali
- Venue:
- VarDial
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 185–190
- Language:
- URL:
- https://aclanthology.org/W18-3921
- DOI:
- Cite (ACL):
- Divyanshu Gupta, Gourav Dhakad, Jayprakash Gupta, and Anil Kumar Singh. 2018. IIT (BHU) System for Indo-Aryan Language Identification (ILI) at VarDial 2018. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 185–190, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
- Cite (Informal):
- IIT (BHU) System for Indo-Aryan Language Identification (ILI) at VarDial 2018 (Gupta et al., VarDial 2018)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/W18-3921.pdf