@inproceedings{gasparovic-etal-2025-assessing,
    title = "Assessing the Accuracy of {AI}-Generated Idiom Translations",
    author = "Gasparovic, Marijana  and
      Brala Vukanovic, Marija  and
      Brkic Bakaric, Marija",
    editor = "Das, Sudhansu Bala  and
      Mishra, Pruthwik  and
      Singh, Alok  and
      Muhammad, Shamsuddeen Hassan  and
      Ekbal, Asif  and
      Das, Uday Kumar",
    booktitle = "Proceedings of the Workshop on Beyond English: Natural Language Processing for all Languages in an Era of Large Language Models",
    month = sep,
    year = "2025",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, BULGARIA",
    url = "https://preview.aclanthology.org/corrections-2026-01/2025.globalnlp-1.15/",
    pages = "131--140",
    abstract = "Idioms pose unique challenges for machine translation due to their metaphorical nature and cultural nuances. Consequently, they often present a translation problem even for humans. This longitudinal study evaluates the performance of ChatGPT in translating idiomatic expressions between English and Croatian, comparing results across two time points. The test set comprises 72 idioms in each translation direction, divided into three categories based on equivalence: complete, partial, and zero, with each category representing one-third of the set. The evaluation considers three layers: translation of the isolated idiom, translation of an online excerpt containing the idiom, and translation of a self-constructed example sentence. As expected, accuracy generally declined with decreasing equivalence. However, a follow-up study conducted six months later highlighted the need for continuous monitoring of machine translation tools."
}