Signing Avatar Performance Evaluation within EASIER Project
Athanasia-Lida Dimou, Vassilis Papavassiliou, John McDonald, Theodore Goulas, Kyriaki Vasilaki, Anna Vacalopoulou, Stavroula-Evita Fotinea, Eleni Efthimiou, Rosalee Wolfe
Abstract
The direct involvement of deaf users in the development and evaluation of signing avatars is imperative to achieve legibility and raise trust among synthetic signing technology consumers. A paradigm of constructive cooperation between researchers and the deaf community is the EASIER project , where user driven design and technology development have already started producing results. One major goal of the project is the direct involvement of sign language (SL) users at every stage of development of the project’s signing avatar. As developers wished to consider every parameter of SL articulation including affect and prosody in developing the EASIER SL representation engine, it was necessary to develop a steady communication channel with a wide public of SL users who may act as evaluators and can provide guidance throughout research steps, both during the project’s end-user evaluation cycles and beyond. To this end, we have developed a questionnaire-based methodology, which enables researchers to reach signers of different SL communities on-line and collect their guidance and preferences on all aspects of SL avatar animation that are under study. In this paper, we report on the methodology behind the application of the EASIER evaluation framework for end-user guidance in signing avatar development as it is planned to address signers of four SLs -Greek Sign Language (GSL), French Sign Language (LSF), German Sign Language (DGS) and Swiss German Sign Language (DSGS)- during the first project evaluation cycle. We also briefly report on some interesting findings from the pilot implementation of the questionnaire with content from the Greek Sign Language (GSL).- Anthology ID:
- 2022.sltat-1.6
- Volume:
- Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives
- Month:
- June
- Year:
- 2022
- Address:
- Marseille, France
- Editors:
- Eleni Efthimiou, Stavroula-Evita Fotinea, Thomas Hanke, John C. McDonald, Dimitar Shterionov, Rosalee Wolfe
- Venue:
- SLTAT
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 39–44
- Language:
- URL:
- https://aclanthology.org/2022.sltat-1.6
- DOI:
- Cite (ACL):
- Athanasia-Lida Dimou, Vassilis Papavassiliou, John McDonald, Theodore Goulas, Kyriaki Vasilaki, Anna Vacalopoulou, Stavroula-Evita Fotinea, Eleni Efthimiou, and Rosalee Wolfe. 2022. Signing Avatar Performance Evaluation within EASIER Project. In Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives, pages 39–44, Marseille, France. European Language Resources Association.
- Cite (Informal):
- Signing Avatar Performance Evaluation within EASIER Project (Dimou et al., SLTAT 2022)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2022.sltat-1.6.pdf