Comparing Kaldi-Based Pipeline Elpis and Whisper for Čakavian Transcription
Austin Jones, Shulin Zhang, John Hale, Margaret Renwick, Zvjezdana Vrzic, Keith Langston
Abstract
Automatic speech recognition (ASR) has the potential to accelerate the documentation of endangered languages, but the dearth of resources poses a major obstacle. Čakavian, an endangered variety spoken primarily in Croatia, is a case in point, lacking transcription tools that could aid documentation efforts. We compare training a new ASR model on a limited dataset using the Kaldi-based ASR pipeline Elpis to using the same dataset to adapt the transformer-based pretrained multilingual model Whisper, to determine which is more practical in the documentation context. Results show that Whisper outperformed Elpis, achieving the lowest average Word Error Rate (WER) of 57.3% and median WER of 35.48%. While Elpis offers a less computationally expensive model and friendlier user experience, Whisper appears better at adapting to our collected Čakavian data.- Anthology ID:
- 2024.fieldmatters-1.8
- Volume:
- Proceedings of the 3rd Workshop on NLP Applications to Field Linguistics (Field Matters 2024)
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Oleg Serikov, Ekaterina Voloshina, Anna Postnikova, Saliha Muradoglu, Eric Le Ferrand, Elena Klyachko, Ekaterina Vylomova, Tatiana Shavrina, Francis Tyers
- Venues:
- FieldMatters | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 61–68
- Language:
- URL:
- https://aclanthology.org/2024.fieldmatters-1.8
- DOI:
- Cite (ACL):
- Austin Jones, Shulin Zhang, John Hale, Margaret Renwick, Zvjezdana Vrzic, and Keith Langston. 2024. Comparing Kaldi-Based Pipeline Elpis and Whisper for Čakavian Transcription. In Proceedings of the 3rd Workshop on NLP Applications to Field Linguistics (Field Matters 2024), pages 61–68, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Comparing Kaldi-Based Pipeline Elpis and Whisper for Čakavian Transcription (Jones et al., FieldMatters-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.fieldmatters-1.8.pdf