TTIC’s Submission to WMT-SLT 23
Marcelo Sandoval-Castaneda, Yanhong Li, Bowen Shi, Diane Brentari, Karen Livescu, Gregory Shakhnarovich
Abstract
In this paper, we describe TTIC’s submission to WMT 2023 Sign Language Translation task on the Swiss-German Sign Language (DSGS) to German track. Our approach explores the advantages of using large-scale self-supervised pre-training in the task of sign language translation, over more traditional approaches that rely heavily on supervision, along with costly labels such as gloss annotations. The proposed model consists of a VideoSwin transformer for image encoding, and a T5 model adapted to receive VideoSwin features as input instead of text. In WMT-SLT 22’s development set, this system achieves 2.03 BLEU score, a 59% increase over the previous best reported performance. In the official test set, our primary submission achieves 1.1 BLEU score and 17.0 chrF score.- Anthology ID:
- 2023.wmt-1.35
- Volume:
- Proceedings of the Eighth Conference on Machine Translation
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Philipp Koehn, Barry Haddow, Tom Kocmi, Christof Monz
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 344–350
- Language:
- URL:
- https://aclanthology.org/2023.wmt-1.35
- DOI:
- 10.18653/v1/2023.wmt-1.35
- Cite (ACL):
- Marcelo Sandoval-Castaneda, Yanhong Li, Bowen Shi, Diane Brentari, Karen Livescu, and Gregory Shakhnarovich. 2023. TTIC’s Submission to WMT-SLT 23. In Proceedings of the Eighth Conference on Machine Translation, pages 344–350, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- TTIC’s Submission to WMT-SLT 23 (Sandoval-Castaneda et al., WMT 2023)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2023.wmt-1.35.pdf