2024
pdf
abs
Comparison of the Intimacy Process between Real and Acting-based Long-term Text Chats
Tsunehiro Arimoto
|
Hiroaki Sugiyama
|
Hiromi Narimatsu
|
Masahiro Mizukami
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Long-term chatbots are expected to develop relationships with users. The major trend in this field’s recent long-term chatbot studies is to train systems with virtual long-term chat data called Multi-Session Chat (MSC), which collects text chat from multiple sessions of crowd workers playing the roles of speakers with defined personas. However, no investigation has attempted to determine whether such virtual long-term chat can successfully simulate relationship-building between speakers. To clarify the difference between an actual long-term intimacy process and an MSC intimacy process, this study collects real long-term chat and MSC in Japanese and compares them in terms of speech form and dialogue acts. The results of analyzing these factors suggest that MSC have an unnatural tendency to behave as if they have a close relationship with non-polite speech levels compared to actual long-term chats, but also as if they have a shallow relationship with more questions than real long-term chats.
2023
pdf
abs
Time-Considerable Dialogue Models via Reranking by Time Dependency
Yuiko Tsunomori
|
Masakazu Ishihata
|
Hiroaki Sugiyama
Findings of the Association for Computational Linguistics: EMNLP 2023
In the last few years, generative dialogue models have shown excellent performance and have been used for various applications. As chatbots become more prevalent in our daily lives, more and more people expect them to behave more like humans, but existing dialogue models do not consider the time information that people are constantly aware of. In this paper, we aim to construct a time-considerable dialogue model that actively utilizes time information. First, we categorize responses by their naturalness at different times and introduce a new metric to classify responses into our categories. Then, we propose a new reranking method to make the existing dialogue model time-considerable using the proposed metric and subjectively evaluate the performances of the obtained time-considerable dialogue models by humans.
2022
pdf
bib
abs
Bipartite-play Dialogue Collection for Practical Automatic Evaluation of Dialogue Systems
Shiki Sato
|
Yosuke Kishinami
|
Hiroaki Sugiyama
|
Reina Akama
|
Ryoko Tokuhisa
|
Jun Suzuki
Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing: Student Research Workshop
Automation of dialogue system evaluation is a driving force for the efficient development of dialogue systems. This paper introduces the bipartite-play method, a dialogue collection method for automating dialogue system evaluation. It addresses the limitations of existing dialogue collection methods: (i) inability to compare with systems that are not publicly available, and (ii) vulnerability to cheating by intentionally selecting systems to be compared. Experimental results show that the automatic evaluation using the bipartite-play method mitigates these two drawbacks and correlates as strongly with human subjectivity as existing methods.
pdf
abs
Citation Sentence Generation Leveraging the Content of Cited Papers
Akito Arita
|
Hiroaki Sugiyama
|
Kohji Dohsaka
|
Rikuto Tanaka
|
Hirotoshi Taira
Proceedings of the Third Workshop on Scholarly Document Processing
We address automatic citation sentence generation, which reduces the burden on writing scientific papers. For highly accurate citation senetence generation, appropriate language must be learned using information such as the relationship between the cited source and the cited paper as well as the context in which the paper cited. Although the abstracts of papers have been used for the generation in the past, they often contain extra information in the citation sentence, which might negatively impact the generation of citation sentences. Therefore, this study attempts to learn a highly accurate citation sentence generation model using sentences from cited articles that resemble the previous sentence to the cited location, thereby utilizing information that is more useful for citation sentence generation.
2020
pdf
abs
Collection and Analysis of Dialogues Provided by Two Speakers Acting as One
Tsunehiro Arimoto
|
Ryuichiro Higashinaka
|
Kou Tanaka
|
Takahito Kawanishi
|
Hiroaki Sugiyama
|
Hiroshi Sawada
|
Hiroshi Ishiguro
Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue
We are studying a cooperation style where multiple speakers can provide both advanced dialogue services and operator education. We focus on a style in which two operators interact with a user by pretending to be a single operator. For two operators to effectively act as one, each must adjust his/her conversational content and timing to the other. In the process, we expect each operator to experience the conversational content of his/her partner as if it were his/her own, creating efficient and effective learning of the other’s skill. We analyzed this educational effect and examined whether dialogue services can be successfully provided by collecting travel guidance dialogue data from operators who give travel information to users. In this paper, we report our preliminary results on dialogue content and user satisfaction of operators and users.
2018
pdf
Collection of Multimodal Dialog Data and Analysis of the Result of Annotation of Users’ Interest Level
Masahiro Araki
|
Sayaka Tomimasu
|
Mikio Nakano
|
Kazunori Komatani
|
Shogo Okada
|
Shinya Fujie
|
Hiroaki Sugiyama
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
2014
pdf
Towards an open-domain conversational system fully based on natural language processing
Ryuichiro Higashinaka
|
Kenji Imamura
|
Toyomi Meguro
|
Chiaki Miyazaki
|
Nozomi Kobayashi
|
Hiroaki Sugiyama
|
Toru Hirano
|
Toshiro Makino
|
Yoshihiro Matsuo
Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers
2013
pdf
Open-domain Utterance Generation for Conversational Dialogue Systems using Web-scale Dependency Structures
Hiroaki Sugiyama
|
Toyomi Meguro
|
Ryuichiro Higashinaka
|
Yasuhiro Minami
Proceedings of the SIGDIAL 2013 Conference