Abstract
Language models such as GPT-3 have caused a furore in the research community. Some studies found that GPT-3 has some creative abilities and makes mistakes that are on par with human behaviour. This paper answers a related question: Who is GPT-3? We administered two validated measurement tools to GPT-3 to assess its personality, the values it holds and its self-reported demographics. Our results show that GPT-3 scores similarly to human samples in terms of personality and - when provided with a model response memory - in terms of the values it holds. We provide the first evidence of psychological assessment of the GPT-3 model and thereby add to our understanding of this language model. We close with suggestions for future research that moves social science closer to language models and vice versa.- Anthology ID:
- 2022.nlpcss-1.24
- Volume:
- Proceedings of the Fifth Workshop on Natural Language Processing and Computational Social Science (NLP+CSS)
- Month:
- November
- Year:
- 2022
- Address:
- Abu Dhabi, UAE
- Editors:
- David Bamman, Dirk Hovy, David Jurgens, Katherine Keith, Brendan O'Connor, Svitlana Volkova
- Venue:
- NLP+CSS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 218–227
- Language:
- URL:
- https://aclanthology.org/2022.nlpcss-1.24
- DOI:
- 10.18653/v1/2022.nlpcss-1.24
- Cite (ACL):
- Marilù Miotto, Nicola Rossberg, and Bennett Kleinberg. 2022. Who is GPT-3? An exploration of personality, values and demographics. In Proceedings of the Fifth Workshop on Natural Language Processing and Computational Social Science (NLP+CSS), pages 218–227, Abu Dhabi, UAE. Association for Computational Linguistics.
- Cite (Informal):
- Who is GPT-3? An exploration of personality, values and demographics (Miotto et al., NLP+CSS 2022)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2022.nlpcss-1.24.pdf