@inproceedings{jiang-etal-2025-meaningful,
    title = "Meaningful Pose-Based Sign Language Evaluation",
    author = {Jiang, Zifan  and
      Leong, Colin  and
      Moryossef, Amit  and
      Cory, Oliver  and
      Ivashechkin, Maksym  and
      Tarigopula, Neha  and
      Zhang, Biao  and
      G{\"o}hring, Anne  and
      Rios, Annette  and
      Sennrich, Rico  and
      Ebling, Sarah},
    editor = "Haddow, Barry  and
      Kocmi, Tom  and
      Koehn, Philipp  and
      Monz, Christof",
    booktitle = "Proceedings of the Tenth Conference on Machine Translation",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.wmt-1.4/",
    pages = "64--80",
    ISBN = "979-8-89176-341-8",
    abstract = "We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based, and back-translation-based metrics. We show tradeoffs between different metrics in different scenarios through (1) automatic meta-evaluation of sign-level retrieval, and (2) a human correlation study of text-to-pose translation across different sign languages. Our findings, along with the open-source pose-evaluation toolkit, provide a practical and reproducible approach for developing and evaluating sign language translation or generation systems."
}