Semi-Automatic Sign Language Corpora Annotation using Lexical Representations of Signs

Matilde Gonzalez, Michael Filhol, Christophe Collet


Abstract
Nowadays many researches focus on the automatic recognition of sign language. High recognition rates are achieved using lot of training data. This data is, generally, collected by manual annotating SL video corpus. However this is time consuming and the results depend on the annotators knowledge. In this work we intend to assist the annotation in terms of glosses which consist on writing down the sign meaning sign for sign thanks to automatic video processing techniques. In this case using learning data is not suitable since at the first step it will be needed to manually annotate the corpus. Also the context dependency of signs and the co-articulation effect in continuous SL make the collection of learning data very difficult. Here we present a novel approach which uses lexical representations of sign to overcome these problems and image processing techniques to match sign performances to sign representations. Signs are described using Zeebede (ZBD) which is a descriptor of signs that considers the high variability of signs. A ZBD database is used to stock signs and can be queried using several characteristics. From a video corpus sequence features are extracted using a robust body part tracking approach and a semi-automatic sign segmentation algorithm. Evaluation has shown the performances and limitation of the proposed approach.
Anthology ID:
L12-1433
Volume:
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
Month:
May
Year:
2012
Address:
Istanbul, Turkey
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2430–2434
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/741_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Matilde Gonzalez, Michael Filhol, and Christophe Collet. 2012. Semi-Automatic Sign Language Corpora Annotation using Lexical Representations of Signs. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 2430–2434, Istanbul, Turkey. European Language Resources Association (ELRA).
Cite (Informal):
Semi-Automatic Sign Language Corpora Annotation using Lexical Representations of Signs (Gonzalez et al., LREC 2012)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/741_Paper.pdf