Developing Classifiers for Affirmative and Negative User Responses with Limited Target Domain Data for Dialogue System Development Tools
Yunosuke Kubo, Ryo Yanagimoto, Mikio Nakano, Kenta Yamamoto, Ryu Takeda, Kazunori Komatani
Abstract
We aim to develop a library for classifying affirmative and negative user responses, intended for integration into a dialogue system development toolkit. Such a library is expected to highly perform even with minimal annotated target domain data, addressing the practical challenge of preparing large datasets for each target domain. This short paper compares several approaches under conditions where little or no annotated data is available in the target domain. One approach involves fine-tuning a pre-trained BERT model, while the other utilizes a GPT API for zero-shot or few-shot learning. Since these approaches differ in execution speed, development effort, and execution costs, in addition to performance, the results serve as a basis for discussing an appropriate configuration suited to specific requirements. Additionally, we have released the training data and the fine-tuned BERT model for Japanese affirmative/negative classification.- Anthology ID:
- 2025.iwsds-1.33
- Volume:
- Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology
- Month:
- May
- Year:
- 2025
- Address:
- Bilbao, Spain
- Editors:
- Maria Ines Torres, Yuki Matsuda, Zoraida Callejas, Arantza del Pozo, Luis Fernando D'Haro
- Venues:
- IWSDS | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 309–317
- Language:
- URL:
- https://preview.aclanthology.org/fix-sig-urls/2025.iwsds-1.33/
- DOI:
- Cite (ACL):
- Yunosuke Kubo, Ryo Yanagimoto, Mikio Nakano, Kenta Yamamoto, Ryu Takeda, and Kazunori Komatani. 2025. Developing Classifiers for Affirmative and Negative User Responses with Limited Target Domain Data for Dialogue System Development Tools. In Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology, pages 309–317, Bilbao, Spain. Association for Computational Linguistics.
- Cite (Informal):
- Developing Classifiers for Affirmative and Negative User Responses with Limited Target Domain Data for Dialogue System Development Tools (Kubo et al., IWSDS 2025)
- PDF:
- https://preview.aclanthology.org/fix-sig-urls/2025.iwsds-1.33.pdf