Abstract
Many common character-level, string-to-string transduction tasks, e.g., grapheme-to-phoneme conversion and morphological inflection, consist almost exclusively of monotonic transduction. Neural sequence-to-sequence models with soft attention, non-monotonic models, outperform popular monotonic models. In this work, we ask the following question: Is monotonicity really a helpful inductive bias in these tasks? We develop a hard attention sequence-to-sequence model that enforces strict monotonicity and learns alignment jointly. With the help of dynamic programming, we are able to compute the exact marginalization over all alignments. Our models achieve state-of-the-art performance on morphological inflection. Furthermore, we find strong performance on two other character-level transduction tasks. Code is available at https://github.com/shijie-wu/neural-transducer.- Anthology ID:
- P19-1148
- Volume:
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Anna Korhonen, David Traum, Lluís Màrquez
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1530–1537
- Language:
- URL:
- https://aclanthology.org/P19-1148
- DOI:
- 10.18653/v1/P19-1148
- Cite (ACL):
- Shijie Wu and Ryan Cotterell. 2019. Exact Hard Monotonic Attention for Character-Level Transduction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1530–1537, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Exact Hard Monotonic Attention for Character-Level Transduction (Wu & Cotterell, ACL 2019)
- PDF:
- https://preview.aclanthology.org/fix-dup-bibkey/P19-1148.pdf
- Code
- shijie-wu/neural-transducer + additional community code