Abstract
Sequential recurrent neural networks have achieved superior performance on language modeling, but overlook the structure information in natural language. Recent works on structure-aware models have shown promising results on language modeling. However, how to incorporate structure knowledge on corpus without syntactic annotations remains an open problem. In this work, we propose neural variational language model (NVLM), which enables the sharing of grammar knowledge among different corpora. Experimental results demonstrate the effectiveness of our framework on two popular benchmark datasets. With the help of shared grammar, our language model converges significantly faster to a lower perplexity on new training corpus.- Anthology ID:
- P19-1437
- Volume:
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2019
- Address:
- Florence, Italy
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 4442–4453
- Language:
- URL:
- https://aclanthology.org/P19-1437
- DOI:
- 10.18653/v1/P19-1437
- Cite (ACL):
- Yuyu Zhang and Le Song. 2019. Language Modeling with Shared Grammar. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4442–4453, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Language Modeling with Shared Grammar (Zhang & Song, ACL 2019)
- PDF:
- https://preview.aclanthology.org/nodalida-main-page/P19-1437.pdf
- Data
- Penn Treebank