Abstract
We incorporate morphological supervision into character language models (CLMs) via multitasking and show that this addition improves bits-per-character (BPC) performance across 24 languages, even when the morphology data and language modeling data are disjoint. Analyzing the CLMs shows that inflected words benefit more from explicitly modeling morphology than uninflected words, and that morphological supervision improves performance even as the amount of language modeling data grows. We then transfer morphological supervision across languages to improve performance in the low-resource setting.- Anthology ID:
- P19-1156
- Volume:
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Anna Korhonen, David Traum, Lluís Màrquez
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1606–1613
- Language:
- URL:
- https://aclanthology.org/P19-1156
- DOI:
- 10.18653/v1/P19-1156
- Cite (ACL):
- Terra Blevins and Luke Zettlemoyer. 2019. Better Character Language Modeling through Morphology. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1606–1613, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Better Character Language Modeling through Morphology (Blevins & Zettlemoyer, ACL 2019)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/P19-1156.pdf