@inproceedings{pedersen-etal-2025-evaluating,
    title = "Evaluating {LLM}-Generated Explanations of Metaphors {--} A Culture-Sensitive Study of {Danish}",
    author = "Pedersen, Bolette S.  and
      S{\o}rensen, Nathalie  and
      Nimb, Sanni  and
      Hansen, Dorte Haltrup  and
      Olsen, Sussi  and
      Al-Laith, Ali",
    editor = "Johansson, Richard  and
      Stymne, Sara",
    booktitle = "Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)",
    month = mar,
    year = "2025",
    address = "Tallinn, Estonia",
    publisher = "University of Tartu Library",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.nodalida-1.51/",
    pages = "470--479",
    ISBN = "978-9908-53-109-0",
    abstract = "In this study, we examine how well Danish culture-specific metaphors are explained by two of the best performing language models for Danish, namely ChatGPT and Llama. For comparison, the explana- tions are measured against how well cross- lingual (or `universal') metaphors are ex- plained by the models; referring here to metaphors that exist in Danish as well as across cultures and languages and in par- ticular in English. To perform our study, we compile a pilot dataset of 150 Danish metaphors and idioms divided tentatively by culture specificity. We prompt the two models and perform a careful qualitative evaluation of the explanations against a four-graded scale. Our studies show that both models are heavily biased towards English since they have much more suc- cess in explaining the metaphors that also exist in English than the culture-specific ones, relying presumably on erroneous transfer from English when dealing with the latter. In particular, the sentiment of the culture-specific metaphors seems to be often `lost in translation'. We further claim that this strong colouring towards English poses a serious problem in the era of LLMs with regards to developing and maintaining cultural and linguistic diver- sity in other languages."
}