Evaluation of Argument Search Approaches in the Context of Argumentative Dialogue Systems
Niklas Rach, Yuki Matsuda, Johannes Daxenberger, Stefan Ultes, Keiichi Yasumoto, Wolfgang Minker
Abstract
We present an approach to evaluate argument search techniques in view of their use in argumentative dialogue systems by assessing quality aspects of the retrieved arguments. To this end, we introduce a dialogue system that presents arguments by means of a virtual avatar and synthetic speech to users and allows them to rate the presented content in four different categories (Interesting, Convincing, Comprehensible, Relation). The approach is applied in a user study in order to compare two state of the art argument search engines to each other and with a system based on traditional web search. The results show a significant advantage of the two search engines over the baseline. Moreover, the two search engines show significant advantages over each other in different categories, thereby reflecting strengths and weaknesses of the different underlying techniques.- Anthology ID:
- 2020.lrec-1.65
- Volume:
- Proceedings of the Twelfth Language Resources and Evaluation Conference
- Month:
- May
- Year:
- 2020
- Address:
- Marseille, France
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 513–522
- Language:
- English
- URL:
- https://aclanthology.org/2020.lrec-1.65
- DOI:
- Cite (ACL):
- Niklas Rach, Yuki Matsuda, Johannes Daxenberger, Stefan Ultes, Keiichi Yasumoto, and Wolfgang Minker. 2020. Evaluation of Argument Search Approaches in the Context of Argumentative Dialogue Systems. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 513–522, Marseille, France. European Language Resources Association.
- Cite (Informal):
- Evaluation of Argument Search Approaches in the Context of Argumentative Dialogue Systems (Rach et al., LREC 2020)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2020.lrec-1.65.pdf