@inproceedings{mihaylov-shtedritski-2024-elegant,
    title = "What an Elegant Bridge: Multilingual {LLM}s are Biased Similarly in Different Languages",
    author = "Mihaylov, Viktor  and
      Shtedritski, Aleksandar",
    editor = "Peled-Cohen, Lotem  and
      Calderon, Nitay  and
      Lissak, Shir  and
      Reichart, Roi",
    booktitle = "Proceedings of the 1st Workshop on NLP for Science (NLP4Science)",
    month = nov,
    year = "2024",
    address = "Miami, FL, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2024.nlp4science-1.3/",
    doi = "10.18653/v1/2024.nlp4science-1.3",
    pages = "16--23",
    abstract = "This paper investigates biases of Large Language Models (LLMs) through the lens of grammatical gender. Drawing inspiration from seminal works in psycholinguistics, particularly the study of gender{'}s influence on language perception, we leverage multilingual LLMs to revisit and expand upon the foundational experiments of Boroditsky (2003). Employing LLMs as a novel method for examining psycholinguistic biases related to grammatical gender, we prompt a model to describe nouns with adjectives in various languages, focusing specifically on languages with grammatical gender. In particular, we look at adjective co-occurrences across gender and languages, and train a binary classifier to predict grammatical gender given adjectives an LLM uses to describe a noun. Surprisingly, we find that a simple classifier can not only predict noun gender above chance but also exhibit cross-language transferability. We show that while LLMs may describe words differently in different languages, they are biased similarly."
}