Modeling and evaluating dialog success in the LAST MINUTE corpus

Dietmar Rösner, Rafael Friesen, Stephan Günther, Rico Andrich

Abstract
The LAST MINUTE corpus comprises records and transcripts of naturalistic problem solving dialogs between N = 130 subjects and a companion system simulated in a Wizard of Oz experiment. Our goal is to detect dialog situations where subjects might break up the dialog with the system which might happen when the subject is unsuccessful. We present a dialog act based representation of the dialog courses in the problem solving phase of the experiment and propose and evaluate measures for dialog success or failure derived from this representation. This dialog act representation refines our previous coarse measure as it enables the correct classification of many dialog sequences that were ambiguous before. The dialog act representation is useful for the identification of different subject groups and the exploration of interesting dialog courses in the corpus. We find young females to be most successful in the challenging last part of the problem solving phase and young subjects to have the initiative in the dialog more often than the elderly.
Anthology ID:
L14-1285
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
259–265
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/321_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Dietmar Rösner, Rafael Friesen, Stephan Günther, and Rico Andrich. 2014. Modeling and evaluating dialog success in the LAST MINUTE corpus. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 259–265, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
Modeling and evaluating dialog success in the LAST MINUTE corpus (Rösner et al., LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/321_Paper.pdf
PDF Search