Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data
Mozhdeh Gheini, Tatiana Likhomanenko, Matthias Sperber, Hendra Setiawan
Abstract
Self-training has been shown to be helpful in addressing data scarcity for many domains, including vision, speech, and language. Specifically, self-training, or pseudo-labeling, labels unsupervised data and adds that to the training pool. In this work, we investigate and use pseudo-labeling for a recently proposed novel setup: joint transcription and translation of speech, which suffers from an absence of sufficient parallel data resources. We show that under such data-deficient circumstances, the unlabeled data can significantly vary in domain from the supervised data, which results in pseudo-label quality degradation. We investigate two categories of remedies that require no additional supervision and target the domain mismatch: pseudo-label filtering and data augmentation. We show that pseudo-label analysis and processing in this way results in additional gains on top of the vanilla pseudo-labeling setup providing a total improvement of up to 0.4% absolute WER and 2.1 BLEU points for En–De and 0.6% absolute WER and 2.2 BLEU points for En–Zh.- Anthology ID:
- 2023.findings-acl.483
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2023
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7637–7650
- Language:
- URL:
- https://aclanthology.org/2023.findings-acl.483
- DOI:
- 10.18653/v1/2023.findings-acl.483
- Cite (ACL):
- Mozhdeh Gheini, Tatiana Likhomanenko, Matthias Sperber, and Hendra Setiawan. 2023. Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7637–7650, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data (Gheini et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2023.findings-acl.483.pdf