@inproceedings{baez-santamaria-etal-2022-evaluating,
    title = "Evaluating Agent Interactions Through Episodic Knowledge Graphs",
    author = "Baez Santamaria, Selene  and
      Vossen, Piek  and
      Baier, Thomas",
    editor = "Lim, Heuiseok  and
      Kim, Seungryong  and
      Lee, Yeonsoo  and
      Lin, Steve  and
      Seo, Paul Hongsuck  and
      Suh, Yumin  and
      Jang, Yoonna  and
      Lim, Jungwoo  and
      Hur, Yuna  and
      Son, Suhyune",
    booktitle = "Proceedings of the 1st Workshop on Customized Chat Grounding Persona and Knowledge",
    month = oct,
    year = "2022",
    address = "Gyeongju, Republic of Korea",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2022.ccgpk-1.3/",
    pages = "15--28",
    abstract = "We present a new method based on episodic Knowledge Graphs (eKGs) for evaluating (multimodal) conversational agents in open domains. This graph is generated by interpreting raw signals during conversation and is able to capture the accumulation of knowledge over time. We apply structural and semantic analysis of the resulting graphs and translate the properties into qualitative measures. We compare these measures with existing automatic and manual evaluation metrics commonly used for conversational agents. Our results show that our Knowledge-Graph-based evaluation provides more qualitative insights into interaction and the agent{'}s behavior."
}