Class-based LSTM Russian Language Model with Linguistic Information

Irina Kipyatkova, Alexey Karpov

Abstract
In the paper, we present class-based LSTM Russian language models (LMs) with classes generated with the use of both word frequency and linguistic information data, obtained with the help of the “VisualSynan” software from the AOT project. We have created LSTM LMs with various numbers of classes and compared them with word-based LM and class-based LM with word2vec class generation in terms of perplexity, training time, and WER. In addition, we performed a linear interpolation of LSTM language models with the baseline 3-gram language model. The LSTM language models were used for very large vocabulary continuous Russian speech recognition at an N-best list rescoring stage. We achieved significant progress in training time reduction with only slight degradation in recognition accuracy comparing to the word-based LM. In addition, our LM with classes generated using linguistic information outperformed LM with classes generated using word2vec. We achieved WER of 14.94 % at our own speech corpus of continuous Russian speech that is 15 % relative reduction with respect to the baseline 3-gram model.
Anthology ID:
2020.lrec-1.300
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
2470–2474
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.300
DOI:
Bibkey:
Cite (ACL):
Irina Kipyatkova and Alexey Karpov. 2020. Class-based LSTM Russian Language Model with Linguistic Information. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2470–2474, Marseille, France. European Language Resources Association.
Cite (Informal):
Class-based LSTM Russian Language Model with Linguistic Information (Kipyatkova & Karpov, LREC 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2020.lrec-1.300.pdf
PDF Search