Interventional Speech Noise Injection for ASR Generalizable Spoken Language Understanding
YeonJoon Jung, Jaeseong Lee, Seungtaek Choi, Dohyeon Lee, Minsoo Kim, Seung-won Hwang
Abstract
Recently, pre-trained language models (PLMs) have been increasingly adopted in spoken language understanding (SLU). However, automatic speech recognition (ASR) systems frequently produce inaccurate transcriptions, leading to noisy inputs for SLU models, which can significantly degrade their performance. To address this, our objective is to train SLU models to withstand ASR errors by exposing them to noises commonly observed in ASR systems, referred to as ASR-plausible noises. Speech noise injection (SNI) methods have pursued this objective by introducing ASR-plausible noises, but we argue that these methods are inherently biased towards specific ASR systems, or ASR-specific noises. In this work, we propose a novel and less biased augmentation method of introducing the noises that are plausible to any ASR system, by cutting off the non-causal effect of noises. Experimental results and analyses demonstrate the effectiveness of our proposed methods in enhancing the robustness and generalizability of SLU models against unseen ASR systems by introducing more diverse and plausible ASR noises in advance.- Anthology ID:
- 2024.emnlp-main.1149
- Volume:
- Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 20642–20655
- Language:
- URL:
- https://preview.aclanthology.org/icon-24-ingestion/2024.emnlp-main.1149/
- DOI:
- 10.18653/v1/2024.emnlp-main.1149
- Cite (ACL):
- YeonJoon Jung, Jaeseong Lee, Seungtaek Choi, Dohyeon Lee, Minsoo Kim, and Seung-won Hwang. 2024. Interventional Speech Noise Injection for ASR Generalizable Spoken Language Understanding. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20642–20655, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- Interventional Speech Noise Injection for ASR Generalizable Spoken Language Understanding (Jung et al., EMNLP 2024)
- PDF:
- https://preview.aclanthology.org/icon-24-ingestion/2024.emnlp-main.1149.pdf