How Gender Interacts with Political Values: A Case Study on Czech BERT Models

Adnan Al Ali, Jindřich Libovický

Abstract
Neural language models, which reach state-of-the-art results on most natural language processing tasks, are trained on large text corpora that inevitably contain value-burdened content and often capture undesirable biases, which the models reflect. This case study focuses on the political biases of pre-trained encoders in Czech and compares them with a representative value survey. Because Czech is a gendered language, we also measure how the grammatical gender coincides with responses to men and women in the survey. We introduce a novel method for measuring the model’s perceived political values. We find that the models do not assign statement probability following value-driven reasoning, and there is no systematic difference between feminine and masculine sentences. We conclude that BERT-sized models do not manifest systematic alignment with political values and that the biases observed in the models are rather due to superficial imitation of training data patterns than systematic value beliefs encoded in the models.
Anthology ID:
2024.lrec-main.719
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
8200–8210
Language:
URL:
https://aclanthology.org/2024.lrec-main.719
DOI:
Bibkey:
Cite (ACL):
Adnan Al Ali and Jindřich Libovický. 2024. How Gender Interacts with Political Values: A Case Study on Czech BERT Models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 8200–8210, Torino, Italia. ELRA and ICCL.
Cite (Informal):
How Gender Interacts with Political Values: A Case Study on Czech BERT Models (Al Ali & Libovický, LREC-COLING 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-2/2024.lrec-main.719.pdf
PDF Search