Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction
Maria Ryskina, Eduard Hovy, Taylor Berg-Kirkpatrick, Matthew R. Gormley
Abstract
Traditionally, character-level transduction problems have been solved with finite-state models designed to encode structural and linguistic knowledge of the underlying process, whereas recent approaches rely on the power and flexibility of sequence-to-sequence models with attention. Focusing on the less explored unsupervised learning scenario, we compare the two model classes side by side and find that they tend to make different types of errors even when achieving comparable performance. We analyze the distributions of different error classes using two unsupervised tasks as testbeds: converting informally romanized text into the native script of its language (for Russian, Arabic, and Kannada) and translating between a pair of closely related languages (Serbian and Bosnian). Finally, we investigate how combining finite-state and sequence-to-sequence models at decoding time affects the output quantitatively and qualitatively.- Anthology ID:
- 2021.sigmorphon-1.22
- Volume:
- Proceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
- Month:
- August
- Year:
- 2021
- Address:
- Online
- Editors:
- Garrett Nicolai, Kyle Gorman, Ryan Cotterell
- Venue:
- SIGMORPHON
- SIG:
- SIGMORPHON
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 198–211
- Language:
- URL:
- https://aclanthology.org/2021.sigmorphon-1.22
- DOI:
- 10.18653/v1/2021.sigmorphon-1.22
- Cite (ACL):
- Maria Ryskina, Eduard Hovy, Taylor Berg-Kirkpatrick, and Matthew R. Gormley. 2021. Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction. In Proceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 198–211, Online. Association for Computational Linguistics.
- Cite (Informal):
- Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction (Ryskina et al., SIGMORPHON 2021)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2021.sigmorphon-1.22.pdf
- Data
- Dakshina, Taiga Corpus