Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values
Yejin Bang, Tiezheng Yu, Andrea Madotto, Zhaojiang Lin, Mona Diab, Pascale Fung
Abstract
Many NLP classification tasks, such as sexism/racism detection or toxicity detection, are based on human values. Yet, human values can vary under diverse cultural conditions. Therefore, we introduce a framework for value-aligned classification that performs prediction based on explicitly written human values in the command. Along with the task, we propose a practical approach that distills value-aligned knowledge from large-scale language models (LLMs) to construct value-aligned classifiers in two steps. First, we generate value-aligned training data from LLMs by prompt-based few-shot learning. Next, we fine-tune smaller classification models with the generated data for the task. Empirical results show that our VA-Models surpass multiple baselines by at least 15.56% on the F1-score, including few-shot learning with OPT-175B and existing text augmentation methods. We suggest that using classifiers with explicit human value input improves both inclusivity & explainability in AI.- Anthology ID:
- 2023.trustnlp-1.27
- Volume:
- Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Anaelia Ovalle, Kai-Wei Chang, Ninareh Mehrabi, Yada Pruksachatkun, Aram Galystan, Jwala Dhamala, Apurv Verma, Trista Cao, Anoop Kumar, Rahul Gupta
- Venue:
- TrustNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 311–325
- Language:
- URL:
- https://aclanthology.org/2023.trustnlp-1.27
- DOI:
- 10.18653/v1/2023.trustnlp-1.27
- Cite (ACL):
- Yejin Bang, Tiezheng Yu, Andrea Madotto, Zhaojiang Lin, Mona Diab, and Pascale Fung. 2023. Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 311–325, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values (Bang et al., TrustNLP 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2023.trustnlp-1.27.pdf