Modeling Intensification for Sign Language Generation: A Computational Approach
Mert Inan, Yang Zhong, Sabit Hassan, Lorna Quandt, Malihe Alikhani
Abstract
End-to-end sign language generation models do not accurately represent the prosody in sign language. A lack of temporal and spatial variations leads to poor-quality generated presentations that confuse human interpreters. In this paper, we aim to improve the prosody in generated sign languages by modeling intensification in a data-driven manner. We present different strategies grounded in linguistics of sign language that inform how intensity modifiers can be represented in gloss annotations. To employ our strategies, we first annotate a subset of the benchmark PHOENIX-14T, a German Sign Language dataset, with different levels of intensification. We then use a supervised intensity tagger to extend the annotated dataset and obtain labels for the remaining portion of it. This enhanced dataset is then used to train state-of-the-art transformer models for sign language generation. We find that our efforts in intensification modeling yield better results when evaluated with automatic metrics. Human evaluation also indicates a higher preference of the videos generated using our model.- Anthology ID:
- 2022.findings-acl.228
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2022
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Smaranda Muresan, Preslav Nakov, Aline Villavicencio
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2897–2911
- Language:
- URL:
- https://aclanthology.org/2022.findings-acl.228
- DOI:
- 10.18653/v1/2022.findings-acl.228
- Cite (ACL):
- Mert Inan, Yang Zhong, Sabit Hassan, Lorna Quandt, and Malihe Alikhani. 2022. Modeling Intensification for Sign Language Generation: A Computational Approach. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2897–2911, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Modeling Intensification for Sign Language Generation: A Computational Approach (Inan et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/fix-dup-bibkey/2022.findings-acl.228.pdf
- Code
- merterm/modeling-intensification-for-slg