@inproceedings{lacruz-etal-2014-cognitive,
    title = "Cognitive demand and cognitive effort in post-editing",
    author = "Lacruz, Isabel  and
      Denkowski, Michael  and
      Lavie, Alon",
    editor = "O'Brien, Sharon  and
      Simard, Michel  and
      Specia, Lucia",
    booktitle = "Proceedings of the 11th Conference of the Association for Machine Translation in the Americas",
    month = oct # " 22-26",
    year = "2014",
    address = "Vancouver, Canada",
    publisher = "Association for Machine Translation in the Americas",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2014.amta-wptp.6/",
    pages = "73--84",
    abstract = "The pause to word ratio, the number of pauses per word in a post-edited MT segment, is an indicator of cognitive effort in post-editing (Lacruz and Shreve, 2014). We investigate how low the pause threshold can reasonably be taken, and we propose that 300 ms is a good choice, as pioneered by Schilperoord (1996). We then seek to identify a good measure of the cognitive demand imposed by MT output on the post-editor, as opposed to the cognitive effort actually exerted by the post-editor during post-editing. Measuring cognitive demand is closely related to measuring MT utility, the MT quality as perceived by the post-editor. HTER, an extrinsic edit to word ratio that does not necessarily correspond to actual edits per word performed by the post-editor, is a well-established measure of MT quality, but it does not comprehensively capture cognitive demand (Koponen, 2012). We investigate intrinsic measures of MT quality, and so of cognitive demand, through edited-error to word metrics. We find that the transfer-error to word ratio predicts cognitive effort better than mechanical-error to word ratio (Koby and Champe, 2013). We identify specific categories of cognitively challenging MT errors whose error to word ratios correlate well with cognitive effort."
}