Abstract
We report an experiment in which we use machine learning to validate the empirical objectivity of a novel annotation taxonomy for behavioral translation data. The HOF taxonomy defines three translation states according to which a human translator can be in a state of Orientation (O), Hesitation (H) or in a Flow state (F). We aim at validating the taxonomy based on a manually annotated dataset that consists of six English-Spanish translation sessions (approx 900 words) and 1813 HOF-annotated Activity Units (AUs). Two annotators annotated the data and obtain high average inter-annotator accuracy 0.76 (kappa 0.88). We train two classifiers, a Multi-layer Perceptron (MLP) and a Random Forest (RF) on the annotated data and tested on held-out data. The classifiers perform well on the annotated data and thus confirm the epistemological objectivity of the annotation taxonomy. Interestingly, inter-classifier accuracy scores are higher than between the two human annotators.- Anthology ID:
- 2024.eamt-1.40
- Volume:
- Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1)
- Month:
- June
- Year:
- 2024
- Address:
- Sheffield, UK
- Editors:
- Carolina Scarton, Charlotte Prescott, Chris Bayliss, Chris Oakley, Joanna Wright, Stuart Wrigley, Xingyi Song, Edward Gow-Smith, Rachel Bawden, Víctor M Sánchez-Cartagena, Patrick Cadwell, Ekaterina Lapshinova-Koltunski, Vera Cabarrão, Konstantinos Chatzitheodorou, Mary Nurminen, Diptesh Kanojia, Helena Moniz
- Venue:
- EAMT
- SIG:
- Publisher:
- European Association for Machine Translation (EAMT)
- Note:
- Pages:
- 480–491
- Language:
- URL:
- https://aclanthology.org/2024.eamt-1.40
- DOI:
- Cite (ACL):
- Michael Carl, Sheng Lu, and Ali Al-Ramadan. 2024. Using Machine Learning to Validate a Novel Taxonomy of Phenomenal Translation States. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 480–491, Sheffield, UK. European Association for Machine Translation (EAMT).
- Cite (Informal):
- Using Machine Learning to Validate a Novel Taxonomy of Phenomenal Translation States (Carl et al., EAMT 2024)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2024.eamt-1.40.pdf