Abstract
Word Error Rate (WER) is a crucial metric for evaluating the performance of automatic speech recognition (ASR) systems. However, its traditional calculation, based on Levenshtein distance, does not account for lexical similarity between words and treats each substitution in a binary manner, while also ignoring segmentation errors. This paper proposes an improvement to WER by introducing a weighted substitution method, based on lexical similarity measures, and incorporating splitting and merging operations to better handle segmentation errors. Unlike other WER variants, our approach is easily integrable and generalizable to various languages, providing a more nuanced and accurate evaluation of ASR transcriptions, particularly for morphologically complex or low-resource languages.- Anthology ID:
- 2024.iwclul-1.3
- Volume:
- Proceedings of the 9th International Workshop on Computational Linguistics for Uralic Languages
- Month:
- November
- Year:
- 2024
- Address:
- Helsinki, Finland
- Editors:
- Mika Hämäläinen, Flammie Pirinen, Melany Macias, Mario Crespo Avila
- Venue:
- IWCLUL
- SIG:
- SIGUR
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 29–35
- Language:
- URL:
- https://preview.aclanthology.org/add_missing_videos/2024.iwclul-1.3/
- DOI:
- Cite (ACL):
- Samy Ouzerrout. 2024. Universal-WER: Enhancing WER with Segmentation and Weighted Substitution for Varied Linguistic Contexts. In Proceedings of the 9th International Workshop on Computational Linguistics for Uralic Languages, pages 29–35, Helsinki, Finland. Association for Computational Linguistics.
- Cite (Informal):
- Universal-WER: Enhancing WER with Segmentation and Weighted Substitution for Varied Linguistic Contexts (Ouzerrout, IWCLUL 2024)
- PDF:
- https://preview.aclanthology.org/add_missing_videos/2024.iwclul-1.3.pdf