Heuristic Hyper-minimization of Finite State Lexicons

Senka Drobac, Krister Lindén, Tommi Pirinen, Miikka Silfverberg

Abstract
Flag diacritics, which are special multi-character symbols executed at runtime, enable optimising finite-state networks by combining identical sub-graphs of its transition graph. Traditionally, the feature has required linguists to devise the optimisations to the graph by hand alongside the morphological description. In this paper, we present a novel method for discovering flag positions in morphological lexicons automatically, based on the morpheme structure implicit in the language description. With this approach, we have gained significant decrease in the size of finite-state networks while maintaining reasonable application speed. The algorithm can be applied to any language description, where the biggest achievements are expected in large and complex morphologies. The most noticeable reduction in size we got with a morphological transducer for Greenlandic, whose original size is on average about 15 times larger than other morphologies. With the presented hyper-minimization method, the transducer is reduced to 10,1% of the original size, with lookup speed decreased only by 9,5%.
Anthology ID:
L14-1609
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
3319–3324
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/784_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Senka Drobac, Krister Lindén, Tommi Pirinen, and Miikka Silfverberg. 2014. Heuristic Hyper-minimization of Finite State Lexicons. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 3319–3324, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
Heuristic Hyper-minimization of Finite State Lexicons (Drobac et al., LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/784_Paper.pdf
PDF Search