Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements
Conrad Borchers, Dalia Gala, Benjamin Gilburt, Eduard Oravkin, Wilfried Bounsi, Yuki M Asano, Hannah Kirk
Abstract
The growing capability and availability of generative language models has enabled a wide range of new downstream tasks. Academic research has identified, quantified and mitigated biases present in language models but is rarely tailored to downstream tasks where wider impact on individuals and society can be felt. In this work, we leverage one popular generative language model, GPT-3, with the goal of writing unbiased and realistic job advertisements. We first assess the bias and realism of zero-shot generated advertisements and compare them to real-world advertisements. We then evaluate prompt-engineering and fine-tuning as debiasing methods. We find that prompt-engineering with diversity-encouraging prompts gives no significant improvement to bias, nor realism. Conversely, fine-tuning, especially on unbiased real advertisements, can improve realism and reduce bias.- Anthology ID:
- 2022.gebnlp-1.22
- Volume:
- Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, Washington
- Editors:
- Christian Hardmeier, Christine Basta, Marta R. Costa-jussà, Gabriel Stanovsky, Hila Gonen
- Venue:
- GeBNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 212–224
- Language:
- URL:
- https://aclanthology.org/2022.gebnlp-1.22
- DOI:
- 10.18653/v1/2022.gebnlp-1.22
- Cite (ACL):
- Conrad Borchers, Dalia Gala, Benjamin Gilburt, Eduard Oravkin, Wilfried Bounsi, Yuki M Asano, and Hannah Kirk. 2022. Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 212–224, Seattle, Washington. Association for Computational Linguistics.
- Cite (Informal):
- Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements (Borchers et al., GeBNLP 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2022.gebnlp-1.22.pdf
- Code
- oxai/gpt3-jobadvert-bias