@inproceedings{li-vu-2024-improving,
    title = "Improving Noisy Student Training for Low-resource Languages in End-to-End {ASR} Using {C}ycle{GAN} and Inter-domain Losses",
    author = "Li, Chia-Yu  and
      Vu, Ngoc Thang",
    editor = "Melero, Maite  and
      Sakti, Sakriani  and
      Soria, Claudia",
    booktitle = "Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://preview.aclanthology.org/fix-sig-urls/2024.sigul-1.17/",
    pages = "133--142",
    abstract = "Training a semi-supervised end-to-end speech recognition system using noisy student training has significantly improved performance. However, this approach requires a substantial amount of paired speech-text and unlabeled speech, which is costly for low-resource languages. Therefore, this paper considers a more extreme case of semi-supervised end-to-end automatic speech recognition where there are limited paired speech-text, unlabeled speech (less than five hours), and abundant external text. Firstly, we observe improved performance by training the model using our previous work on semi-supervised learning ``CycleGAN and inter-domain losses'' solely with external text. Secondly, we enhance ``CycleGAN and inter-domain losses'' by incorporating automatic hyperparameter tuning, calling ``enhanced CycleGAN inter-domain losses.'' Thirdly, we integrate it into the noisy student training approach pipeline for low-resource scenarios. Our experimental results, conducted on six non-English languages from Voxforge and Common Voice, show a 20{\%} word error rate reduction compared to the baseline teacher model and a 10{\%} word error rate reduction compared to the baseline best student model, highlighting the significant improvements achieved through our proposed method."
}