@inproceedings{cui-etal-2020-mutual,
    title = "{M}u{T}ual: A Dataset for Multi-Turn Dialogue Reasoning",
    author = "Cui, Leyang  and
      Wu, Yu  and
      Liu, Shujie  and
      Zhang, Yue  and
      Zhou, Ming",
    editor = "Jurafsky, Dan  and
      Chai, Joyce  and
      Schluter, Natalie  and
      Tetreault, Joel",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2020.acl-main.130/",
    doi = "10.18653/v1/2020.acl-main.130",
    pages = "1406--1416",
    abstract = "Non-task oriented dialogue systems have achieved great success in recent years due to largely accessible conversation data and the development of deep learning techniques. Given a context, current systems are able to yield a relevant and fluent response, but sometimes make logical mistakes because of weak reasoning capabilities. To facilitate the conversation reasoning research, we introduce MuTual, a novel dataset for Multi-Turn dialogue Reasoning, consisting of 8,860 manually annotated dialogues based on Chinese student English listening comprehension exams. Compared to previous benchmarks for non-task oriented dialogue systems, MuTual is much more challenging since it requires a model that be able to handle various reasoning problems. Empirical results show that state-of-the-art methods only reach 71{\%}, which is far behind human performance of 94{\%}, indicating that there is ample room for improving reasoning ability."
}