Evaluation of Argument Search Approaches in the Context of Argumentative Dialogue Systems

Niklas Rach, Yuki Matsuda, Johannes Daxenberger, Stefan Ultes, Keiichi Yasumoto, Wolfgang Minker

Abstract
We present an approach to evaluate argument search techniques in view of their use in argumentative dialogue systems by assessing quality aspects of the retrieved arguments. To this end, we introduce a dialogue system that presents arguments by means of a virtual avatar and synthetic speech to users and allows them to rate the presented content in four different categories (Interesting, Convincing, Comprehensible, Relation). The approach is applied in a user study in order to compare two state of the art argument search engines to each other and with a system based on traditional web search. The results show a significant advantage of the two search engines over the baseline. Moreover, the two search engines show significant advantages over each other in different categories, thereby reflecting strengths and weaknesses of the different underlying techniques.
Anthology ID:
2020.lrec-1.65
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
513–522
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.65
DOI:
Bibkey:
Cite (ACL):
Niklas Rach, Yuki Matsuda, Johannes Daxenberger, Stefan Ultes, Keiichi Yasumoto, and Wolfgang Minker. 2020. Evaluation of Argument Search Approaches in the Context of Argumentative Dialogue Systems. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 513–522, Marseille, France. European Language Resources Association.
Cite (Informal):
Evaluation of Argument Search Approaches in the Context of Argumentative Dialogue Systems (Rach et al., LREC 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2020.lrec-1.65.pdf
PDF Search