Andrea Schröter


2025

pdf bib
Universal Patterns of Grammatical Gender in Multilingual Large Language Models
Andrea Schröter | Ali Basirat
Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025)

Grammatical gender is a fundamental linguistic feature that varies across languages, and its cross-linguistic correspondence has been a central question in disciplines such as cognitive science and linguistic typology. This study takes an information-theoretic approach to investigate the extent to which variational usable information about grammatical gender encoded by a large language model generalizes across languages belonging to different language families. Using mBERT as a case study, we analyze how grammatical gender is encoded and transferred across languages based on the usable information of the intermediate representations. The empirical results provide evidence that gender mechanisms are driven by abstract semantic features largely shared across languages, and that the information becomes more accessible at the higher layers of the language model.