Tü-CL at SIGMORPHON 2023: Straight-Through Gradient Estimation for Hard Attention

Leander Girrbach


Abstract
This paper describes our systems participating in the 2023 SIGMORPHON Shared Task on Morphological Inflection and in the 2023 SIGMORPHON Shared Task on Interlinear Glossing. We propose methods to enrich predictions from neural models with discrete, i.e. interpretable, information. For morphological inflection, our models learn deterministic mappings from subsets of source lemma characters and morphological tags to individual target characters, which introduces interpretability. For interlinear glossing, our models learn a shallow morpheme segmentation in an unsupervised way jointly with predicting glossing lines. Estimated segmentation may be useful when no ground-truth segmentation is available. As both methods introduce discreteness into neural models, our technical contribution is to show that straight-through gradient estimators are effective to train hard attention models.
Anthology ID:
2023.sigmorphon-1.17
Volume:
Proceedings of the 20th SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Garrett Nicolai, Eleanor Chodroff, Frederic Mailhot, Çağrı Çöltekin
Venue:
SIGMORPHON
SIG:
SIGMORPHON
Publisher:
Association for Computational Linguistics
Note:
Pages:
151–165
Language:
URL:
https://aclanthology.org/2023.sigmorphon-1.17
DOI:
10.18653/v1/2023.sigmorphon-1.17
Bibkey:
Cite (ACL):
Leander Girrbach. 2023. Tü-CL at SIGMORPHON 2023: Straight-Through Gradient Estimation for Hard Attention. In Proceedings of the 20th SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 151–165, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Tü-CL at SIGMORPHON 2023: Straight-Through Gradient Estimation for Hard Attention (Girrbach, SIGMORPHON 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-2/2023.sigmorphon-1.17.pdf