Conversational Tutoring in VR Training: The Role of Game Context and State Variables
Maia Aguirre, Ariane Méndez, Aitor García-Pablos, Montse Cuadros, Arantza del Pozo, Oier Lopez de Lacalle, Ander Salaberria, Jeremy Barnes, Pablo Martínez, Muhammad Zeshan Afzal
Abstract
Virtual Reality (VR) training provides safe, cost-effective engagement with lifelike scenarios but lacks intuitive communication between users and the virtual environment. This study investigates the use of Large Language Models (LLMs) as conversational tutors in VR health and safety training, examining the impact of game context and state variables on LLM-generated answers in zero- and few-shot settings. Results demonstrate that incorporating both game context and state information significantly improves answer accuracy, with human evaluations showing gains of up to 0.26 points in zero-shot and 0.18 points in few-shot settings on a 0-1 scale.- Anthology ID:
- 2025.iwsds-1.23
- Volume:
- Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology
- Month:
- May
- Year:
- 2025
- Address:
- Bilbao, Spain
- Editors:
- Maria Ines Torres, Yuki Matsuda, Zoraida Callejas, Arantza del Pozo, Luis Fernando D'Haro
- Venues:
- IWSDS | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 215–224
- Language:
- URL:
- https://preview.aclanthology.org/fix-sig-urls/2025.iwsds-1.23/
- DOI:
- Cite (ACL):
- Maia Aguirre, Ariane Méndez, Aitor García-Pablos, Montse Cuadros, Arantza del Pozo, Oier Lopez de Lacalle, Ander Salaberria, Jeremy Barnes, Pablo Martínez, and Muhammad Zeshan Afzal. 2025. Conversational Tutoring in VR Training: The Role of Game Context and State Variables. In Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology, pages 215–224, Bilbao, Spain. Association for Computational Linguistics.
- Cite (Informal):
- Conversational Tutoring in VR Training: The Role of Game Context and State Variables (Aguirre et al., IWSDS 2025)
- PDF:
- https://preview.aclanthology.org/fix-sig-urls/2025.iwsds-1.23.pdf