Real Time Korean Sign Language (KSL) Translator using Action Recognition

Abstract

This paper presents a real-time Korean Sign Language (KSL) recognition system for individuals with speech and hearing impairments. The system uses the Mediapipe framework and Tensorflow to detect hand gestures and track hand movements. Object detection is used to detect the signer's face and hands, and the system then recognizes and translates the gestures into text in real-time. The proposed system has the potential to improve communication between people with speech and hearing impairments and others who do not understand sign language. Additionally, the system can be extended to recognize other sign languages with appropriate training data. The implementation of this system in real-world scenarios can lead to a more inclusive and accessible environment for people with speech and hearing impairments. The proposed system has the potential to greatly improve communication between individuals with speech and hearing impairments and the wider community, and can be extended to other sign languages with appropriate training data.