@inproceedings{jon-bojar-2024-analysis,
    title = "An Analysis of Surprisal Uniformity in Machine and Human Translations",
    author = "Jon, Josef  and
      Bojar, Ond{\v{r}}ej",
    editor = "Vanroy, Bram  and
      Lefer, Marie-Aude  and
      Macken, Lieve  and
      Ruffo, Paola",
    booktitle = "Proceedings of the 1st Workshop on Creative-text Translation and Technology",
    month = jun,
    year = "2024",
    address = "Sheffield, United Kingdom",
    publisher = "European Association for Machine Translation",
    url = "https://preview.aclanthology.org/fix-sig-urls/2024.ctt-1.5/",
    pages = "40--56",
    abstract = "This study examines neural machine translation (NMT) and its performance on texts that diverege from typical standards, focusing on how information is organized within sentences. We analyze surprisal distributions in source texts, human translations, and machine translations across several datasets to determine if NMT systems naturally promote a uniform density of surprisal in their translations, even when the original texts do not adhere to this principle.The findings reveal that NMT tends to align more closely with source texts in terms of surprisal uniformity compared to human translations.We analyzed absolute values of the surprisal uniformity measures as well, expecting that human translations will be less uniform. In contradiction to our initial hypothesis, we did not find comprehensive evidence for this claim, with some results suggesting this might be the case for very diverse texts, like poetry."
}