Mobile Sign Language Translation System

Views:
 
     
 

Presentation Description

Mobile Sign Language Translation System For Deaf Community (C). Mehrez Boulares, Mohamed Jemni.

Comments

Presentation Transcript

MOBILE SIGN LANGUAGE TRANSLATION SYSTEM FOR DEAF COMMUNITY :

MOBILE SIGN LANGUAGE TRANSLATION SYSTEM FOR DEAF COMMUNITY Mehrez Boulares Mohamed Jemni Research Laboratory of Technologies of Information and Communication & Electrical Engineering ( LaTICE ) University of Tunis 1

Summary:

Summary Motivation Objective Previous work Our approach Conclusion & future works 2

Motivation:

Motivation In the world, there are around 70 million people with hearing deficiencies (information from World Federation of the Deaf http://www.wfdeaf.org/ ). The learning progress is extremely slow and the gain of experience collected by deaf children in four years is equivalent to the gain of one year for hearing children. 3

Motivation:

Motivation Deaf or profoundly deaf people may wear no hearing aid. Some will be able to lip read and understand you nearly perfectly. Some may have problems with verb tenses, concordances of gender and number Many communicate with sign language rather than with written and spoken language. 4

Motivation:

Motivation Today web technologies are almost indispensable in our daily life. We use web technologies as data and content sharing, as a social platform for generating, repositioning and consuming content. Web technologies encompass a variety of different meanings that include new ways of interacting with Web-based applications, collaborative approach and new alternatives to enrich the learning process. 5

Motivation:

Motivation Recent research (Albert L. H. 2009) showing the benefits of collaborative learning (Whitney, G., all, 2011) is being conducted across disciplines and looks at the use of web and virtual world technologies to improve the classical learning ways. 6

Objective:

Objective Collaborative approach based on web and virtual world technologies. Virtual reality description X3D and signs creation (multi-communities creation interface). To overcome traditional video limitations related to bandwidth constraints and video merging problem. 7

Objective:

Objective We are based on web services to create signs and to provide sign language translation. Web services can be used by smart phones and this increases the user area and provides anywhere access to our translation service. Sign language learning tool. 8

Previous work:

Previous work Video support such as sign language video recognition. Deployment of Virtual Signers on the Web: Two main techniques are employed: * pre-synthesized animation and generated animation. => based on motion capture pre-recorded animation using avatar technology. expensive material + decreases the user interactivity to create new signs 9

Previous work:

Previous work *Automatic and real-time generation of animations. eSIGN ,: sending motion command signSMITH : gesture builder No sharing of the created signs to be used by the others communities. No real time translation. No mobile access 10

Our approach:

Our approach 11

Our approach:

Our approach Our web service architecture is divided into three principal parts. The first part is devoted to provide the SML animation related to each word in the sentence. The second part ensures the automatic generation of SML animation to 3D rendering animation. The third part is devoted to convert SML to video animation. 12

Our approach:

Our approach Sign Modeling Language SML: The Sign Modelling Language SML . 13

Our approach:

Our approach 14 Conforme to H-ANIME standard

Our approach:

Our approach SML animation : Send the text from a mobile terminal as http request. Our web service engine generates the SML animation according to each word on the sentence. 15

Our approach:

Our approach 3D rendering: The 3D rendering module interprets the SML description of signs and converts it to 3D rendering animation using our virtual agent. All steps in our system are completely automated. 16

Our approach:

Our approach Sign creation: http://www.youtube.com/watch?v=UNe3yqVach8&feature=youtu.be 17

Our approach:

Our approach Mobile solution and technical details : Mobile phones have very similar features compared to personal computers. Users can access easily to a web service for sign language translation anywhere. 18

Our approach:

Our approach Mobile access to our web service and android OS. Easy to use interface with assisted auto-completion text area which provides existing signs on our remote database. This application has two text input mode.: - Written text. - Received short text messages SMS in smart phone. 19

Our approach:

Our approach The already existing animation will be directly sent as a multimedia URL to mobile terminal. New requests will be rendered and sent as a multimedia streaming URL. =>increases the reactivity rate of our mobile application. 20

Our approach:

Our approach The service response time depends mainly on mobile network and 3D rendering machine power. i5 processor, 4GB RAM ,ATI RADEON HD 5650 graphics card. 5 new words (signs) : 25 seconds this means average of 5 seconds per sign. =>More powerful machine we can reduce the rendering time to reach the real time translation. 21

Conclusion and future works:

Conclusion and future works This paper described a web service solution for sign language translation with mobile access. Incorporates two principal aspects: - Creating signs with our web tool to be stored as SML notation system + Collaborative approach. In the second aspect, we described how our web service translates text to sign language animation. - Different parts of our web service - Mobile access from smart phone + real time translation. 22

Conclusion and future works:

Conclusion and future works This system can be used as an : - Assistant to learn sign language to children or adult persons. - Allows translation of received short text message to sign language animation. As a perspective of our work: - Build an interface which allows hard of hearing people to communicate with hearing people by converting animation to text. - Improve application reactivity to reach real time translation. 23

authorStream Live Help