Jack Bandy
2026
Caged Birds and Cute Bookworms: Feminine Tropes and Implicit Gender Bias in Large Language Models
Sachita Nishal | Jack Bandy
Proceedings of the Workshop on Evaluating Evaluations (EvalEval)
Sachita Nishal | Jack Bandy
Proceedings of the Workshop on Evaluating Evaluations (EvalEval)
This paper introduces a curated dataset for diagnosing implicit gender bias through feminine tropes in narratives generated by large language models. Drawing from a crowd-sourced database of tropes from television media, we create prompts that elicit narratives from LLMs based on historically gendered tropes. We find that LLMs tend to revert to feminine characters in these narratives, even when prompted without explicit gender references, and also when prompted with non-binary (“they/them”) gender references for the main character. In some cases, even when prompted with masculine pronouns (“he/him”), LLMs still use feminine pronouns to describe the main character. The paper describes our dataset creation process and the evaluation of four open-weight models. We discuss implications for future research in mitigating implicit gender bias and its associated representational harms in LLMs, as well as the complex relationship between language models and societal values.