@inproceedings{hsu-etal-2025-improving,
    title = "Improving Low-Resource Speech Recognition with Whisper-{M}o{E} and Synthetic Data Augmentation: A Case Study on {H}akka",
    author = "Hsu, Yuan-Chi  and
      Fang, Liang-Chun  and
      Dai, Hong-Jie",
    editor = "Chang, Kai-Wei  and
      Lu, Ke-Han  and
      Yang, Chih-Kai  and
      Tam, Zhi-Rui  and
      Chang, Wen-Yu  and
      Wang, Chung-Che",
    booktitle = "Proceedings of the 37th Conference on Computational Linguistics and Speech Processing (ROCLING 2025)",
    month = nov,
    year = "2025",
    address = "National Taiwan University, Taipei City, Taiwan",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/dashboard/2025.rocling-main.51/",
    pages = "446--449",
    ISBN = "979-8-89176-379-1",
    abstract = "The objective of this study is to improve speech recognition performance for low-resource Hakka, a language spoken by a specific ethnic group. Our team conducted experiments by fine-tuning different base versions of Whisper (e.g., the original model and the Mandarin-focused Belle model). We found that fine-tuning on different bases yielded distinct advantages and varying results in Hakka character and phonetic recognition tasks. To further enhance model accuracy, we experimented with replacing the q, k, and v linear layers in the attention blocks of the Whisper encoder with a mixture-of-experts model combined with RoLA. In addition, we augmented the training data with synthesized speech generated with diverse voice styles and varying speaking rates. The results showed a 0.73{\%} reduction in character error rate for Task 1 and a 0.2{\%} reduction in word error rate for Task 2. These findings confirm that both architectural adjustments to the model and the strategic use of limited synthetic speech data in low-resource dialect corpora can effectively improve recognition performance."
}