WITcHCRafT: A Workbench for Intelligent exploraTion of Human ComputeR conversaTions

Alexander Schmitt, Gregor Bertrand, Tobias Heinroth, Wolfgang Minker, Jackson Liscombe

Abstract
We present Witchcraft, an open-source framework for the evaluation of prediction models for spoken dialogue systems based on interaction logs and audio recordings. The use of Witchcraft is two fold: first, it provides an adaptable user interface to easily manage and browse thousands of logged dialogues (e.g. calls). Second, with help of the underlying models and the connected machine learning framework RapidMiner the workbench is able to display at each dialogue turn the probability of the task being completed based on the dialogue history. It estimates the emotional state, gender and age of the user. While browsing through a logged conversation, the user can directly observe the prediction result of the models at each dialogue step. By that, Witchcraft allows for spotting problematic dialogue situations and demonstrates where the current system and the prediction models have design flaws. Witchcraft will be made publically available to the community and will be deployed as open-source project.
Anthology ID:
L10-1078
Volume:
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)
Month:
May
Year:
2010
Address:
Valletta, Malta
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2010/pdf/123_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Alexander Schmitt, Gregor Bertrand, Tobias Heinroth, Wolfgang Minker, and Jackson Liscombe. 2010. WITcHCRafT: A Workbench for Intelligent exploraTion of Human ComputeR conversaTions. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), Valletta, Malta. European Language Resources Association (ELRA).
Cite (Informal):
WITcHCRafT: A Workbench for Intelligent exploraTion of Human ComputeR conversaTions (Schmitt et al., LREC 2010)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2010/pdf/123_Paper.pdf
PDF Search