Communicative strategies and patterns of multimodal integration in a speech-to-speech translation system

Susanne Burger, Erica Costantini, Fabio Pianesi


Abstract
When multilingual communication through a speech-to-speech translation system is supported by multimodal features, e.g. pen-based gestures, the following issues arise concerning the nature of the supported communication: a) to what extend does multilingual communication differ from ‘ordinary’ monolingual communication with respect to the dialogue structure and the communicative strategies used by participants; b) the patterns of integration between speech and gestures. Building on the outcomes of a previous work, we present results from a study aimed at addressing those issues. The initial findings confirm that multilingual communication, and the way in which it is realized by actual systems (e.g., with or without the push-to-talk mode) affects the form and structure of the conversation.
Anthology ID:
2003.mtsummit-papers.5
Volume:
Proceedings of Machine Translation Summit IX: Papers
Month:
September 23-27
Year:
2003
Address:
New Orleans, USA
Venue:
MTSummit
SIG:
Publisher:
Note:
Pages:
Language:
URL:
https://aclanthology.org/2003.mtsummit-papers.5
DOI:
Bibkey:
Cite (ACL):
Susanne Burger, Erica Costantini, and Fabio Pianesi. 2003. Communicative strategies and patterns of multimodal integration in a speech-to-speech translation system. In Proceedings of Machine Translation Summit IX: Papers, New Orleans, USA.
Cite (Informal):
Communicative strategies and patterns of multimodal integration in a speech-to-speech translation system (Burger et al., MTSummit 2003)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2003.mtsummit-papers.5.pdf