Simulating Identity, Propagating Bias: Abstraction and Stereotypes in LLM-Generated Text

Pia Sommerauer, Giulia Rambelli, Tommaso Caselli


Abstract
Persona-prompting is a growing strategy to steer LLMs toward simulating particular perspectives or linguistic styles through the lens of a specified identity. While this method is often used to personalize outputs, its impact on how LLMs represent social groups remains underexplored. In this paper, we investigate whether persona-prompting leads to different levels of linguistic abstraction—an established marker of stereotyping—when generating short texts linking socio-demographic categories with stereotypical or non-stereotypical attributes. Drawing on the Linguistic Expectancy Bias framework, we analyze outputs from six open-weight LLMs under three prompting conditions, comparing 11 persona-driven responses to those of a generic AI assistant. To support this analysis, we introduce Self-Stereo, a new dataset of self-reported stereotypes from Reddit. We measure abstraction through three metrics: concreteness, specificity, and negation. Our results highlight the limits of persona-prompting in modulating abstraction in language, confirming criticisms about the ecology of personas as representative of socio-demographic groups and raising concerns about the risk of propagating stereotypes even when seemingly evoking the voice of a marginalized groups.
Anthology ID:
2025.findings-emnlp.1080
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2025
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
19812–19831
Language:
URL:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1080/
DOI:
10.18653/v1/2025.findings-emnlp.1080
Bibkey:
Cite (ACL):
Pia Sommerauer, Giulia Rambelli, and Tommaso Caselli. 2025. Simulating Identity, Propagating Bias: Abstraction and Stereotypes in LLM-Generated Text. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 19812–19831, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Simulating Identity, Propagating Bias: Abstraction and Stereotypes in LLM-Generated Text (Sommerauer et al., Findings 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1080.pdf
Checklist:
 2025.findings-emnlp.1080.checklist.pdf