Abstract
Scaling dense PCFGs to thousands of nonterminals via low-rank parameterizations of the rule probability tensor has been shown to be beneficial for unsupervised parsing. However, PCFGs scaled this way still perform poorly as a language model, and even underperform similarly-sized HMMs. This work introduces SimplePCFG, a simple PCFG formalism with independent left and right productions. Despite imposing a stronger independence assumption than the low-rank approach, we find that this formalism scales more effectively both as a language model and as an unsupervised parser. We further introduce FlashInside, a hardware IO-aware implementation of the inside algorithm for efficiently scaling simple PCFGs. Through extensive experiments on multiple grammar induction benchmarks, we validate the effectiveness of simple PCFGs over low-rank baselines.- Anthology ID:
- 2023.findings-emnlp.113
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1662–1669
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.113
- DOI:
- 10.18653/v1/2023.findings-emnlp.113
- Cite (ACL):
- Wei Liu, Songlin Yang, Yoon Kim, and Kewei Tu. 2023. Simple Hardware-Efficient PCFGs with Independent Left and Right Productions. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1662–1669, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Simple Hardware-Efficient PCFGs with Independent Left and Right Productions (Liu et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2023.findings-emnlp.113.pdf