Exploring the Impact of Modalities on Building Common Ground Using the Collaborative Scene Reconstruction Task

Yosuke Ujigawa, Asuka Shiotani, Masato Takizawa, Eisuke Midorikawa, Ryuichiro Higashinaka, Kazunori Takashio


Abstract
To deepen our understanding of verbal and non-verbal modalities in establishing common ground, this study introduces a novel “collaborative scene reconstruction task.” In this task, pairs of participants, each provided with distinct image sets derived from the same video, work together to reconstruct the sequence of the original video. The level of agreement between the participants on the image order—quantified using Kendall’s rank correlation coefficient—serves as a measure of common ground construction. This approach enables the analysis of how various modalities contribute to the constraction of common ground. A corpus comprising 40 dialogues from 20 participants was collected and analyzed. The findings suggest that specific gestures play a significant role in fostering common ground, offering valuable insights for the development of dialogue systems that leverage multimodal information to enhance user the counstraction of common ground.
Anthology ID:
2025.iwsds-1.2
Volume:
Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology
Month:
May
Year:
2025
Address:
Bilbao, Spain
Editors:
Maria Ines Torres, Yuki Matsuda, Zoraida Callejas, Arantza del Pozo, Luis Fernando D'Haro
Venues:
IWSDS | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
12–19
Language:
URL:
https://preview.aclanthology.org/landing_page/2025.iwsds-1.2/
DOI:
Bibkey:
Cite (ACL):
Yosuke Ujigawa, Asuka Shiotani, Masato Takizawa, Eisuke Midorikawa, Ryuichiro Higashinaka, and Kazunori Takashio. 2025. Exploring the Impact of Modalities on Building Common Ground Using the Collaborative Scene Reconstruction Task. In Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology, pages 12–19, Bilbao, Spain. Association for Computational Linguistics.
Cite (Informal):
Exploring the Impact of Modalities on Building Common Ground Using the Collaborative Scene Reconstruction Task (Ujigawa et al., IWSDS 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2025.iwsds-1.2.pdf