Sign Language Recognition and Translation: A Multi-Modal Approach Using Computer Vision and Natural Language Processing
Jacky Li, Jaren Gerdes, James Gojit, Austin Tao, Samyak Katke, Kate Nguyen, Benyamin Ahmadnia
Abstract
Sign-to-Text (S2T) is a hand gesture recognition program in the American Sign Language (ASL) domain. The primary objective of S2T is to classify standard ASL alphabets and custom signs and convert the classifications into a stream of text using neural networks. This paper addresses the shortcomings of pure Computer Vision techniques and applies Natural Language Processing (NLP) as an additional layer of complexity to increase S2T’s robustness.- Anthology ID:
- 2023.ranlp-1.71
- Volume:
- Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing
- Month:
- September
- Year:
- 2023
- Address:
- Varna, Bulgaria
- Editors:
- Ruslan Mitkov, Galia Angelova
- Venue:
- RANLP
- SIG:
- Publisher:
- INCOMA Ltd., Shoumen, Bulgaria
- Note:
- Pages:
- 658–665
- Language:
- URL:
- https://aclanthology.org/2023.ranlp-1.71
- DOI:
- Cite (ACL):
- Jacky Li, Jaren Gerdes, James Gojit, Austin Tao, Samyak Katke, Kate Nguyen, and Benyamin Ahmadnia. 2023. Sign Language Recognition and Translation: A Multi-Modal Approach Using Computer Vision and Natural Language Processing. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 658–665, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
- Cite (Informal):
- Sign Language Recognition and Translation: A Multi-Modal Approach Using Computer Vision and Natural Language Processing (Li et al., RANLP 2023)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2023.ranlp-1.71.pdf