Abstract
Pose estimation keypoints are widely used in sign language recognition (SLR) as a means of generalising to unseen signers. Despite the advantages of keypoints, SLR models struggle to achieve high recognition accuracy for many signed languages due to the large degree of variability between occurrences of the same signs, the lack of large datasets and the imbalanced nature of the data therein. In this paper we seek to provide a deeper analysis into the ways that these keypoints are used by models in order to determine which are most informative to SLR, identify potentially redundant ones and investigate whether keypoints that are central to differentiating signs in practice are being effectively used as expected by models.- Anthology ID:
- 2024.lrec-main.1387
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 15970–15975
- Language:
- URL:
- https://aclanthology.org/2024.lrec-main.1387
- DOI:
- Cite (ACL):
- Ruth M. Holmes, Ellen Rushe, and Anthony Ventresque. 2024. The Key Points: Using Feature Importance to Identify Shortcomings in Sign Language Recognition Models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15970–15975, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- The Key Points: Using Feature Importance to Identify Shortcomings in Sign Language Recognition Models (Holmes et al., LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/2024.lrec-main.1387.pdf