The Negochat Corpus of Human-agent Negotiation Dialogues

Vasily Konovalov, Ron Artstein, Oren Melamud, Ido Dagan

Abstract
Annotated in-domain corpora are crucial to the successful development of dialogue systems of automated agents, and in particular for developing natural language understanding (NLU) components of such systems. Unfortunately, such important resources are scarce. In this work, we introduce an annotated natural language human-agent dialogue corpus in the negotiation domain. The corpus was collected using Amazon Mechanical Turk following the ‘Wizard-Of-Oz’ approach, where a ‘wizard’ human translates the participants’ natural language utterances in real time into a semantic language. Once dialogue collection was completed, utterances were annotated with intent labels by two independent annotators, achieving high inter-annotator agreement. Our initial experiments with an SVM classifier show that automatically inferring such labels from the utterances is far from trivial. We make our corpus publicly available to serve as an aid in the development of dialogue systems for negotiation agents, and suggest that analogous corpora can be created following our methodology and using our available source code. To the best of our knowledge this is the first publicly available negotiation dialogue corpus.
Anthology ID:
L16-1501
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
3141–3145
Language:
URL:
https://aclanthology.org/L16-1501
DOI:
Bibkey:
Cite (ACL):
Vasily Konovalov, Ron Artstein, Oren Melamud, and Ido Dagan. 2016. The Negochat Corpus of Human-agent Negotiation Dialogues. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 3141–3145, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
The Negochat Corpus of Human-agent Negotiation Dialogues (Konovalov et al., LREC 2016)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/L16-1501.pdf
PDF Search