Using Computer Vision to Analyze Non-manual Marking of Questions in KRSL
Anna Kuznetsova, Alfarabi Imashev, Medet Mukushev, Anara Sandygulova, Vadim Kimmelman
Abstract
This paper presents a study that compares non-manual markers of polar and wh-questions to statements in Kazakh-Russian Sign Language (KRSL) in a dataset collected for NLP tasks. The primary focus of the study is to demonstrate the utility of computer vision solutions for the linguistic analysis of non-manuals in sign languages, although additional corrections are required to account for biases in the output. To this end, we analyzed recordings of 10 triplets of sentences produced by 9 native signers using both manual annotation and computer vision solutions (such as OpenFace). We utilize and improve the computer vision solution, and briefly describe the results of the linguistic analysis.- Anthology ID:
- 2021.mtsummit-at4ssl.6
- Volume:
- Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL)
- Month:
- August
- Year:
- 2021
- Address:
- Virtual
- Editor:
- Dimitar Shterionov
- Venue:
- MTSummit
- SIG:
- Publisher:
- Association for Machine Translation in the Americas
- Note:
- Pages:
- 49–59
- Language:
- URL:
- https://aclanthology.org/2021.mtsummit-at4ssl.6
- DOI:
- Cite (ACL):
- Anna Kuznetsova, Alfarabi Imashev, Medet Mukushev, Anara Sandygulova, and Vadim Kimmelman. 2021. Using Computer Vision to Analyze Non-manual Marking of Questions in KRSL. In Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL), pages 49–59, Virtual. Association for Machine Translation in the Americas.
- Cite (Informal):
- Using Computer Vision to Analyze Non-manual Marking of Questions in KRSL (Kuznetsova et al., MTSummit 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2021.mtsummit-at4ssl.6.pdf