Abstract
Social bias in language - towards genders, ethnicities, ages, and other social groups - poses a problem with ethical impact for many NLP applications. Recent research has shown that machine learning models trained on respective data may not only adopt, but even amplify the bias. So far, however, little attention has been paid to bias in computational argumentation. In this paper, we study the existence of social biases in large English debate portals. In particular, we train word embedding models on portal-specific corpora and systematically evaluate their bias using WEAT, an existing metric to measure bias in word embeddings. In a word co-occurrence analysis, we then investigate causes of bias. The results suggest that all tested debate corpora contain unbalanced and biased data, mostly in favor of male people with European-American names. Our empirical insights contribute towards an understanding of bias in argumentative data sources.- Anthology ID:
- 2020.argmining-1.9
- Volume:
- Proceedings of the 7th Workshop on Argument Mining
- Month:
- December
- Year:
- 2020
- Address:
- Online
- Editors:
- Elena Cabrio, Serena Villata
- Venue:
- ArgMining
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 76–87
- Language:
- URL:
- https://aclanthology.org/2020.argmining-1.9
- DOI:
- Cite (ACL):
- Maximilian Spliethöver and Henning Wachsmuth. 2020. Argument from Old Man’s View: Assessing Social Bias in Argumentation. In Proceedings of the 7th Workshop on Argument Mining, pages 76–87, Online. Association for Computational Linguistics.
- Cite (Informal):
- Argument from Old Man’s View: Assessing Social Bias in Argumentation (Spliethöver & Wachsmuth, ArgMining 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2020.argmining-1.9.pdf