Abstract
The paper describes the evaluation of the KomParse system. KomParse is a dialogue system embedded in a 3-D massive multiplayer online game, allowing conversations between non player characters (NPCs) and game users. In a field test with game users, the system was evaluated with respect to acceptability and usability of the overall system as well as task completion, dialogue control and efficiency of three conversational tasks. Furthermore, subjective feedback has been collected for evaluating the single communication components of the system such as natural language understanding. The results are very satisfying and promising. In general, both the usability and acceptability tests show that the tested NPC is useful and well-accepted by the users. Even if the NPC does not always understand the users well and expresses things unexpected, he could still provide appropriate responses to help users to solve their problems or entertain them.- Anthology ID:
- L12-1419
- Volume:
- Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
- Month:
- May
- Year:
- 2012
- Address:
- Istanbul, Turkey
- Editors:
- Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association (ELRA)
- Note:
- Pages:
- 3535–3542
- Language:
- URL:
- http://www.lrec-conf.org/proceedings/lrec2012/pdf/721_Paper.pdf
- DOI:
- Cite (ACL):
- Tina Kluewer, Feiyu Xu, Peter Adolphs, and Hans Uszkoreit. 2012. Evaluation of the KomParse Conversational Non-Player Characters in a Commercial Virtual World. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 3535–3542, Istanbul, Turkey. European Language Resources Association (ELRA).
- Cite (Informal):
- Evaluation of the KomParse Conversational Non-Player Characters in a Commercial Virtual World (Kluewer et al., LREC 2012)
- PDF:
- http://www.lrec-conf.org/proceedings/lrec2012/pdf/721_Paper.pdf