Muhammad Zeshan Afzal


2025

pdf bib
Conversational Tutoring in VR Training: The Role of Game Context and State Variables
Maia Aguirre | Ariane Méndez | Aitor García-Pablos | Montse Cuadros | Arantza del Pozo | Oier Lopez de Lacalle | Ander Salaberria | Jeremy Barnes | Pablo Martínez | Muhammad Zeshan Afzal
Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology

Virtual Reality (VR) training provides safe, cost-effective engagement with lifelike scenarios but lacks intuitive communication between users and the virtual environment. This study investigates the use of Large Language Models (LLMs) as conversational tutors in VR health and safety training, examining the impact of game context and state variables on LLM-generated answers in zero- and few-shot settings. Results demonstrate that incorporating both game context and state information significantly improves answer accuracy, with human evaluations showing gains of up to 0.26 points in zero-shot and 0.18 points in few-shot settings on a 0-1 scale.