Multimodal Corpora for Silent Speech Interaction

João Freitas, António Teixeira, Miguel Dias

Abstract
A Silent Speech Interface (SSI) allows for speech communication to take place in the absence of an acoustic signal. This type of interface is an alternative to conventional Automatic Speech Recognition which is not adequate for users with some speech impairments or in the presence of environmental noise. The work presented here produces the conditions to explore and analyze complex combinations of input modalities applicable in SSI research. By exploring non-invasive and promising modalities, we have selected the following sensing technologies used in human-computer interaction: Video and Depth input, Ultrasonic Doppler sensing and Surface Electromyography. This paper describes a novel data collection methodology where these independent streams of information are synchronously acquired with the aim of supporting research and development of a multimodal SSI. The reported recordings were divided into two rounds: a first one where the acquired data was silently uttered and a second round where speakers pronounced the scripted prompts in an audible and normal tone. In the first round of recordings, a total of 53.94 minutes were captured where 30.25% was estimated to be silent speech. In the second round of recordings, a total of 30.45 minutes were obtained and 30.05% of the recordings were audible speech.
Anthology ID:
L14-1243
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
4507–4511
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/264_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
João Freitas, António Teixeira, and Miguel Dias. 2014. Multimodal Corpora for Silent Speech Interaction. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 4507–4511, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
Multimodal Corpora for Silent Speech Interaction (Freitas et al., LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/264_Paper.pdf
PDF Search