@article{sarti-etal-2025-qe4pe,
    title = "{QE}4{PE}: Word-level Quality Estimation for Human Post-Editing",
    author = "Sarti, Gabriele  and
      Zouhar, Vil{\'e}m  and
      Chrupa{\l}a, Grzegorz  and
      Guerberof-Arenas, Ana  and
      Nissim, Malvina  and
      Bisazza, Arianna",
    journal = "Transactions of the Association for Computational Linguistics",
    volume = "13",
    year = "2025",
    address = "Cambridge, MA",
    publisher = "MIT Press",
    url = "https://preview.aclanthology.org/ingest-eacl/2025.tacl-1.64/",
    doi = "10.1162/tacl.a.46",
    pages = "1410--1435",
    abstract = "Word-level quality estimation (QE) methods aim to detect erroneous spans in machine translations, which can direct and facilitate human post-editing. While the accuracy of word-level QE systems has been assessed extensively, their usability and downstream influence on the speed, quality, and editing choices of human post-editing remain understudied. In this study, we investigate the impact of word-level QE on machine translation (MT) post-editing in a realistic setting involving 42 professional post-editors across two translation directions. We compare four error-span highlight modalities, including supervised and uncertainty-based word-level QE methods, for identifying potential errors in the outputs of a state-of-the-art neural MT model. Post-editing effort and productivity are estimated from behavioral logs, while quality improvements are assessed by word- and segment-level human annotation. We find that domain, language and editors' speed are critical factors in determining highlights' effectiveness, with modest differences between human-made and automated QE highlights underlining a gap between accuracy and usability in professional workflows."
}