Nadine Braun
2022
A reproduction study of methods for evaluating dialogue system output: Replicating Santhanam and Shaikh (2019)
Anouck Braggaar
|
Frédéric Tomas
|
Peter Blomsma
|
Saar Hommes
|
Nadine Braun
|
Emiel van Miltenburg
|
Chris van der Lee
|
Martijn Goudbeek
|
Emiel Krahmer
Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges
In this paper, we describe our reproduction ef- fort of the paper: Towards Best Experiment Design for Evaluating Dialogue System Output by Santhanam and Shaikh (2019) for the 2022 ReproGen shared task. We aim to produce the same results, using different human evaluators, and a different implementation of the automatic metrics used in the original paper. Although overall the study posed some challenges to re- produce (e.g. difficulties with reproduction of automatic metrics and statistics), in the end we did find that the results generally replicate the findings of Santhanam and Shaikh (2019) and seem to follow similar trends.
2016
The Multilingual Affective Soccer Corpus (MASC): Compiling a biased parallel corpus on soccer reportage in English, German and Dutch
Nadine Braun
|
Martijn Goudbeek
|
Emiel Krahmer
Proceedings of the 9th International Natural Language Generation conference
Search
Co-authors
- Martijn Goudbeek 2
- Emiel Krahmer 2
- Anouck Braggaar 1
- Frédéric Tomas 1
- Peter Blomsma 1
- show all...
Venues
- inlg2