Chien-Chun Wang


2026

Low-resource automatic speech recognition remains a critical challenge due to the scarcity of transcribed data for many languages.Taiwanese Hokkien exemplifies this problem as, although extensive speech content exists in television dramas and online videos, transcriptions are scarce and most available subtitles are in Mandarin.To address this gap, this paper presents TG-ASR for Taiwanese drama speech recognition, a translation-guided ASR framework that leverages multilingual translation embeddings to enhance recognition in low-resource conditions.The framework centers on the parallel gated cross-attention (PGCA) mechanism, which adaptively integrates embeddings from multiple auxiliary languages into the ASR decoder.This mechanism enables robust cross-linguistic semantic guidance while maintaining stable optimization and avoiding interference between languages.To support future research, we release YT-THDC, a 30-hour corpus of Taiwanese drama speech with aligned Mandarin subtitles and manually verified Taiwanese transcriptions.Extensive experiments and analysis identify which auxiliary languages most effectively improve Taiwanese ASR, achieving a 13.51% relative reduction in character error rate and demonstrating the potential of translation-guided learning for underrepresented languages in real-world scenarios.

2025

Automatic speech recognition (ASR) for low-resource languages such as Taiwanese Hokkien is difficult due to the scarcity of annotated data. However, direct fine-tuning on Han-character transcriptions often fails to capture detailed phonetic and tonal cues, while training only on romanization lacks lexical and syntactic coverage. In addition, prior studies have rarely explored staged strategies that integrate both annotation types. To address this gap, we present CLiFT-ASR, a cross-lingual fine-tuning framework that builds on Mandarin HuBERT models and progressively adapts them to Taiwanese Hokkien. The framework employs a two-stage process in which it first learns acoustic and tonal representations from phonetic Tai-lo annotations and then captures vocabulary and syntax from Han-character transcriptions. This progressive adaptation enables effective alignment between speech sounds and orthographic structures. Experiments on the TAT-MOE corpus demonstrate that CLiFT-ASR achieves a 24.88% relative reduction in character error rate (CER) compared with strong baselines. The results indicate that CLiFT-ASR provides an effective and parameter-efficient solution for Taiwanese Hokkien ASR and that it has potential to benefit other low-resource language scenarios.