The Paradigm Discovery Problem
Alexander Erdmann, Micha Elsner, Shijie Wu, Ryan Cotterell, Nizar Habash
Abstract
This work treats the paradigm discovery problem (PDP), the task of learning an inflectional morphological system from unannotated sentences. We formalize the PDP and develop evaluation metrics for judging systems. Using currently available resources, we construct datasets for the task. We also devise a heuristic benchmark for the PDP and report empirical results on five diverse languages. Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm. Then, we bootstrap a neural transducer on top of the clustered data to predict words to realize the empty paradigm slots. An error analysis of our system suggests clustering by cell across different inflection classes is the most pressing challenge for future work.- Anthology ID:
- 2020.acl-main.695
- Volume:
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2020
- Address:
- Online
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7778–7790
- Language:
- URL:
- https://aclanthology.org/2020.acl-main.695
- DOI:
- 10.18653/v1/2020.acl-main.695
- Cite (ACL):
- Alexander Erdmann, Micha Elsner, Shijie Wu, Ryan Cotterell, and Nizar Habash. 2020. The Paradigm Discovery Problem. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7778–7790, Online. Association for Computational Linguistics.
- Cite (Informal):
- The Paradigm Discovery Problem (Erdmann et al., ACL 2020)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/2020.acl-main.695.pdf
- Code
- alexerdmann/ParadigmDiscovery
- Data
- Universal Dependencies