@inproceedings{hollain-etal-2023-analyzing,
    title = "Analyzing the Potential of Linguistic Features for Sign Spotting: A Look at Approximative Features",
    author = "Hollain, Natalie  and
      Larson, Martha  and
      Roelofsen, Floris",
    editor = "Shterionov, Dimitar  and
      Sisto, Mirella De  and
      Muller, Mathias  and
      Landuyt, Davy Van  and
      Omardeen, Rehana  and
      Oboyle, Shaun  and
      Braffort, Annelies  and
      Roelofsen, Floris  and
      Blain, Fred  and
      Vanroy, Bram  and
      Avramidis, Eleftherios",
    booktitle = "Proceedings of the Second International Workshop on Automatic Translation for Signed and Spoken Languages",
    month = jun,
    year = "2023",
    address = "Tampere, Finland",
    publisher = "European Association for Machine Translation",
    url = "https://aclanthology.org/2023.at4ssl-1.1",
    pages = "1--10",
    abstract = "Sign language processing is the field of research that aims to recognize, retrieve, and spot signs in videos. Various approaches have been developed, varying in whether they use linguistic features and whether they use landmark detection tools or not. Incorporating linguistics holds promise for improving sign language processing in terms of performance, generalizability, and explainability. This paper focuses on the task of sign spotting and aims to expand on the approximative linguistic features that have been used in previous work, and to understand when linguistic features deliver an improvement over landmark features. We detect landmarks with Mediapipe and extract linguistically relevant features from them, including handshape, orientation, location, and movement. We compare a sign spotting model using linguistic features with a model operating on landmarks directly, finding that the approximate linguistic features tested in this paper capture some aspects of signs better than the landmark features, while they are worse for others.",
}