Abstract
Lingua is an application developed for the Church of Jesus Christ of Latter-day Saints that performs both real-time interpretation of live speeches and automatic video dubbing (AVD). Like other AVD systems, it can perform synchronized automatic dubbing, given video files and optionally, corresponding text files using a traditional ASR–MT–TTS pipeline. Lingua’s unique contribution is that it can also operate in real-time with a slight delay of a few seconds to interpret live speeches. If no source-language script is provided, the translations are exactly as recognized by ASR and translated by MT. If a script is provided, Lingua matches the recognized ASR segments with script segments and passes the latter to MT for translation and subsequent TTS. If a human translation is also provided, it is passed directly to TTS. Lingua switches between these modes dynamically, enabling translation of off-script comments and different levels of quality for multiple languages. (see extended abstract)- Anthology ID:
- 2022.amta-upg.14
- Volume:
- Proceedings of the 15th Biennial Conference of the Association for Machine Translation in the Americas (Volume 2: Users and Providers Track and Government Track)
- Month:
- September
- Year:
- 2022
- Address:
- Orlando, USA
- Editors:
- Janice Campbell, Stephen Larocca, Jay Marciano, Konstantin Savenkov, Alex Yanishevsky
- Venue:
- AMTA
- SIG:
- Publisher:
- Association for Machine Translation in the Americas
- Note:
- Pages:
- 202–209
- Language:
- URL:
- https://aclanthology.org/2022.amta-upg.14
- DOI:
- Cite (ACL):
- Nathan Anderson, Caleb Wilson, and Stephen D. Richardson. 2022. Lingua: Addressing Scenarios for Live Interpretation and Automatic Dubbing. In Proceedings of the 15th Biennial Conference of the Association for Machine Translation in the Americas (Volume 2: Users and Providers Track and Government Track), pages 202–209, Orlando, USA. Association for Machine Translation in the Americas.
- Cite (Informal):
- Lingua: Addressing Scenarios for Live Interpretation and Automatic Dubbing (Anderson et al., AMTA 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.amta-upg.14.pdf