@inproceedings{muller-eberstein-etal-2025-dakultur,
    title = "{D}a{K}ultur: Evaluating the Cultural Awareness of Language Models for {D}anish with Native Speakers",
    author = {M{\"u}ller-Eberstein, Max  and
      Zhang, Mike  and
      Bassignana, Elisa  and
      Trolle, Peter Brunsgaard  and
      Goot, Rob Van Der},
    editor = "Prabhakaran, Vinodkumar  and
      Dev, Sunipa  and
      Benotti, Luciana  and
      Hershcovich, Daniel  and
      Cao, Yong  and
      Zhou, Li  and
      Cabello, Laura  and
      Adebara, Ife",
    booktitle = "Proceedings of the 3rd Workshop on Cross-Cultural Considerations in NLP (C3NLP 2025)",
    month = may,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.c3nlp-1.5/",
    pages = "50--58",
    ISBN = "979-8-89176-237-4",
    abstract = "Large Language Models (LLMs) have seen widespread societal adoption. However, while they are able to interact with users in languages beyond English, they have been shown to lack cultural awareness, providing anglocentric or inappropriate responses for underrepresented language communities. To investigate this gap and disentangle linguistic versus cultural proficiency, we conduct the first cultural evaluation study for the mid-resource language of Danish, in which native speakers prompt different models to solve tasks requiring cultural awareness. Our analysis of the resulting 1,038 interactions from 63 demographically diverse participants highlights open challenges to cultural adaptation: Particularly, how currently employed automatically translated data are insufficient to train or measure cultural adaptation, and how training on native-speaker data can more than double response acceptance rates. We release our study data as DaKultur - the first native Danish cultural awareness dataset."
}