@inproceedings{hamanaka-etal-2026-sensor,
    title = "Sensor-Augmented Voice Activity Projection for Enhancing Turn-Taking Prediction",
    author = "Hamanaka, Satoki  and
      Kishino, Yasue  and
      Tsunomori, Yuiko  and
      Mizutani, Shin  and
      Chiba, Yuya  and
      Okoshi, Tadashi  and
      Nakazawa, Jin",
    editor = "Choi, Jinho D.  and
      Chen, Yun-Nung  and
      Funakoshi, Kotaro  and
      Emami, Ali",
    booktitle = "Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue",
    month = aug,
    year = "2026",
    address = "Atlanta, Georgia, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/cawl-year/2026.sigdial-1.12/",
    pages = "164--169",
    abstract = "Voice Activity Projection (VAP) has been actively studied to enable natural turn-taking in spoken dialogue systems, relying primarily on acoustic features. Visual cues such as head movements are also known to contribute to turn-taking prediction; however, camera-based approaches are affected by placement and lighting conditions and are not always reliably available to dialogue systems. As a camera-independent approach for directly capturing head motion, earable devices offer a promising solution. In this study, we propose Sensor-Augmented VAP, a framework that integrates in-ear inertial measurement unit (IMU) signals with a pre-trained VAP model via a lightweight residual fusion module. To validate our proposed method, we collected a dataset pairing conversational audio with in-ear IMU data, comprising 12 dyadic Japanese dialogues recorded using microphones and earbuds. Experiments in speaker-independent and speaker-dependent settings demonstrate that IMU fusion consistently improves weighted F1 score for shift detection and reduces VAP loss over the audio-only baseline. These results confirm that head-motion cues are effective for enhancing turn-taking prediction."
}