Unsupervised Induction of Linguistic Categories with Records of Reading, Speaking, and Writing

Maria Barrett, Ana Valeria González-Garduño, Lea Frermann, Anders Søgaard


Abstract
When learning POS taggers and syntactic chunkers for low-resource languages, different resources may be available, and often all we have is a small tag dictionary, motivating type-constrained unsupervised induction. Even small dictionaries can improve the performance of unsupervised induction algorithms. This paper shows that performance can be further improved by including data that is readily available or can be easily obtained for most languages, i.e., eye-tracking, speech, or keystroke logs (or any combination thereof). We project information from all these data sources into shared spaces, in which the union of words is represented. For English unsupervised POS induction, the additional information, which is not required at test time, leads to an average error reduction on Ontonotes domains of 1.5% over systems augmented with state-of-the-art word embeddings. On Penn Treebank the best model achieves 5.4% error reduction over a word embeddings baseline. We also achieve significant improvements for syntactic chunk induction. Our analysis shows that improvements are even bigger when the available tag dictionaries are smaller.
Anthology ID:
N18-1184
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2028–2038
Language:
URL:
https://aclanthology.org/N18-1184
DOI:
10.18653/v1/N18-1184
Bibkey:
Cite (ACL):
Maria Barrett, Ana Valeria González-Garduño, Lea Frermann, and Anders Søgaard. 2018. Unsupervised Induction of Linguistic Categories with Records of Reading, Speaking, and Writing. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2028–2038, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Unsupervised Induction of Linguistic Categories with Records of Reading, Speaking, and Writing (Barrett et al., NAACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/N18-1184.pdf
Data
Penn Treebank