@inproceedings{nakano-etal-2025-methodology,
    title = "A Methodology for Identifying Evaluation Items for Practical Dialogue Systems Based on Business-Dialogue System Alignment Models",
    author = "Nakano, Mikio  and
      Takeuchi, Hironori  and
      Komatani, Kazunori",
    editor = "Torres, Maria Ines  and
      Matsuda, Yuki  and
      Callejas, Zoraida  and
      del Pozo, Arantza  and
      D'Haro, Luis Fernando",
    booktitle = "Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology",
    month = may,
    year = "2025",
    address = "Bilbao, Spain",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.iwsds-1.24/",
    pages = "225--237",
    ISBN = "979-8-89176-248-0",
    abstract = "This paper proposes a methodology for identifying evaluation items for practical dialogue systems. Traditionally, user satisfaction and user experiences have been the primary metrics for evaluating dialogue systems. However, there are various other evaluation items to consider when developing and operating practical dialogue systems, and such evaluation items are expected to lead to new research topics. So far, there has been no methodology for identifying these evaluation items. We propose identifying evaluation items based on business-dialogue system alignment models, which are applications of business-IT alignment models used in the development and operation of practical IT systems. We also present a generic model that facilitates the construction of a business-dialogue system alignment model for each dialogue system."
}