ZeroStance: Leveraging ChatGPT for Open-Domain Stance Detection via Dataset Generation

Chenye Zhao, Yingjie Li, Cornelia Caragea, Yue Zhang


Abstract
Zero-shot stance detection that aims to detect the stance (typically against, favor, or neutral) towards unseen targets has attracted considerable attention. However, most previous studies only focus on targets from a single or limited text domains (e.g., financial domain), and thus zero-shot models cannot generalize well to unseen targets of diverse domains (e.g., political domain). In this paper, we consider a more realistic task, i.e., open-domain stance detection, which aims at training a model that is able to generalize well to unseen targets across multiple domains of interest. Particularly, we propose a novel dataset generation method ZeroStance, which leverages ChatGPT to construct a synthetic open-domain dataset CHATStance that covers a wide range of domains. We then train an open-domain model on our synthetic dataset after proper data filtering. Extensive results indicate that our model, when trained on this synthetic dataset, shows superior generalization to unseen targets of diverse domains over baselines on most benchmarks. Our method requires only a task description in the form of a prompt and is much more cost-effective and data-efficient than previous methods. We will release our code and data to facilitate future research.
Anthology ID:
2024.findings-acl.794
Volume:
Findings of the Association for Computational Linguistics: ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
13390–13405
Language:
URL:
https://preview.aclanthology.org/build-pipeline-with-new-library/2024.findings-acl.794/
DOI:
10.18653/v1/2024.findings-acl.794
Bibkey:
Cite (ACL):
Chenye Zhao, Yingjie Li, Cornelia Caragea, and Yue Zhang. 2024. ZeroStance: Leveraging ChatGPT for Open-Domain Stance Detection via Dataset Generation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 13390–13405, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
ZeroStance: Leveraging ChatGPT for Open-Domain Stance Detection via Dataset Generation (Zhao et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/build-pipeline-with-new-library/2024.findings-acl.794.pdf