@inproceedings{budimir-2025-challenge,
    title = "The Challenge of Translating Culture-Specific Items: Evaluating {MT} and {LLM}s Compared to Human Translators",
    author = "Budimir, Bojana",
    editor = "Bouillon, Pierrette  and
      Gerlach, Johanna  and
      Girletti, Sabrina  and
      Volkart, Lise  and
      Rubino, Raphael  and
      Sennrich, Rico  and
      Farinha, Ana C.  and
      Gaido, Marco  and
      Daems, Joke  and
      Kenny, Dorothy  and
      Moniz, Helena  and
      Szoc, Sara",
    booktitle = "Proceedings of Machine Translation Summit XX: Volume 1",
    month = jun,
    year = "2025",
    address = "Geneva, Switzerland",
    publisher = "European Association for Machine Translation",
    url = "https://preview.aclanthology.org/mtsummit-25-ingestion/2025.mtsummit-1.36/",
    pages = "455--467",
    ISBN = "978-2-9701897-0-1",
    abstract = "We evaluate state-of-the-art Large Language Models (LLM{'}s) ChatGPT-4o, Gemini 1.5 Flash, and Google Translate, by focusing on the translation of culture-specific items (CSIs) between an underrepresented language pair: the Flemish variant of Dutch and Serbian. Using a corpus derived from three Flemish novels we analyze CSIs in three cultural domains: Material Culture, Proper Names, and Social Culture. Translation strategies are examined on a spectrum that goes from conservation to substitution. Quantitative analysis explores strategy distribution, while qualitative analysis investigates errors, linguistic accuracy, and cultural adaptation. Despite advancements, models struggle to balance cultural nuances with understandability for the target readers. Gemini aligns most closely with human translation strategies, while Google Translate shows significant limitations. These findings underscore the challenges of translating CSIs{---}particularly Proper Names{---}in low-resource languages and offer insights for improving machine translation models."
}