Jiawen Deng
2022
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark
Hao Sun
|
Guangxuan Xu
|
Jiawen Deng
|
Jiale Cheng
|
Chujie Zheng
|
Hao Zhou
|
Nanyun Peng
|
Xiaoyan Zhu
|
Minlie Huang
Findings of the Association for Computational Linguistics: ACL 2022
Dialogue safety problems severely limit the real-world deployment of neural conversational models and have attracted great research interests recently. However, dialogue safety problems remain under-defined and the corresponding dataset is scarce. We propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings, with focuses on context-sensitive unsafety, which is under-explored in prior works. To spur research in this direction, we compile DiaSafety, a dataset with rich context-sensitive unsafe examples. Experiments show that existing safety guarding tools fail severely on our dataset. As a remedy, we train a dialogue safety classifier to provide a strong baseline for context-sensitive dialogue unsafety detection. With our classifier, we perform safety evaluations on popular conversational models and show that existing dialogue systems still exhibit concerning context-sensitive safety problems.
Search
Co-authors
- Hao Sun 1
- Guangxuan Xu 1
- Jiale Cheng 1
- Chujie Zheng 1
- Hao Zhou 1
- show all...