Semi-Supervised Induction of POS-Tag Lexicons with Tree Models

Maciej Janicki

Abstract
We approach the problem of POS tagging of morphologically rich languages in a setting where only a small amount of labeled training data is available. We show that a bigram HMM tagger benefits from re-training on a larger untagged text using Baum-Welch estimation. Most importantly, this estimation can be significantly improved by pre-guessing tags for OOV words based on morphological criteria. We consider two models for this task: a character-based recurrent neural network, which guesses the tag from the string form of the word, and a recently proposed graph-based model of morphological transformations. In the latter, the unknown POS tags can be modeled as latent variables in a way very similar to Hidden Markov Tree models and an analogue of the Forward-Backward algorithm can be formulated, which enables us to compute expected values over unknown taggings. We evaluate both the quality of the induced tag lexicon and its impact on the HMM’s tagging accuracy. In both tasks, the graph-based morphology model performs significantly better than the RNN predictor. This confirms the intuition that morphologically related words provide useful information about an unknown word’s POS tag.
Anthology ID:
R19-1060
Volume:
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)
Month:
September
Year:
2019
Address:
Varna, Bulgaria
Editors:
Ruslan Mitkov, Galia Angelova
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
507–515
Language:
URL:
https://aclanthology.org/R19-1060
DOI:
10.26615/978-954-452-056-4_060
Bibkey:
Cite (ACL):
Maciej Janicki. 2019. Semi-Supervised Induction of POS-Tag Lexicons with Tree Models. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 507–515, Varna, Bulgaria. INCOMA Ltd..
Cite (Informal):
Semi-Supervised Induction of POS-Tag Lexicons with Tree Models (Janicki, RANLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/ml4al-ingestion/R19-1060.pdf
PDF Search