Title:
The VirtualSign Channel for the Communication Between Deaf and Hearing Users [Download]Authors:
Oliveira, Tiago and Escudeiro, Nuno and Escudeiro, Paula and Rocha, Emanuel and Barbosa, Fernando Maciel
Index Terms:
Assistive technologies;Deafness;Sign language;Sign language;sign language recognition;translation;deaf;hearing disabilities;accessibility;inclusion
Abstract:
Deaf students, who use sign language as their mother language, continuously experience difficulties to communicate with non-deaf in their daily lives. This is a severe handicap in education settings seriously jeopardizing deaf people chances to progress in their professional career. Deaf people's comprehension of texts is limited due to grammar differences between sign and oral languages. There is a need to improve the communication between deaf and non-deaf and to support deaf students in environments where they are unable to be accompanied by sign interpreters. This article details the improvements and current structure of the VirtualSign platform, a bidirectional sign language to text translation tool in development since 2015. The platform has two main components, sign to text and text to sign, that are both described. Translation from text to sign relies on a 3D avatar. Translation from sign to text relies on a set of data gloves and Kinect. In this paper we discuss the relevance of different types of data gloves. VirtualSign is being developed in cooperation with the deaf communities from six different European countries and Brazil. This solution to support deaf students in educational settings has received positive feedback on several tests and pilot experiments. Some planned improvements and future functionalities for the tool are also mentioned and detailed.
DOI:
How to cite:
Oliveira, Tiago and Escudeiro, Nuno and Escudeiro, Paula and Rocha, Emanuel and Barbosa, Fernando Maciel, "The VirtualSign Channel for the Communication Between Deaf and Hearing Users" in IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, pp. 188-195, Nov. 2019. doi: 10.1109/RITA.2019.2952270