Translator of Indonesian Sign Language Video using Convolutional Neural Network with Transfer Learning

Shania, Sesilia and Naufal, Mohammad Farid and Prasetyo, Vincentius Riandaru and Azmi, Sanusi (2022) Translator of Indonesian Sign Language Video using Convolutional Neural Network with Transfer Learning. Indonesian Journal of Information Systems (IJIS), 5 (1). pp. 17-27. ISSN 2623-2308(Online); 2623-0119(Print)

PDF Vincentius Riandaru_Translator of Indonesian Sign Language Video.pdf Download (1MB)

Abstract

Sign language is a language used to communicate by utilizing gestures and facial expressions. This study focuses on classification of Bahasa Isyarat Indonesia (BISINDO). There are still many people who have difficulty communicating with the deaf people. This study buildt video-based translator system using Convolutional Neural Network (CNN) with transfer learning which was commonly used in computer vision especially in image classification. Transfer learning used in this study were a MobileNetV2, ResNet50V2, and Xception. This study applied 11 different commonly used vocabularies in BISINDO. The predictions were made in a real-time scenario using a webcam. In addition, the system would also ease the interaction approach between deaf and normal people. From all the experiments, it was found that the Xception architectures has the best F1 Score of 98.5%.

Item Type:	Article
Uncontrolled Keywords:	BISINDO; CNN; Translator; Sign Language
Subjects:	Q Science > QA Mathematics > QA76 Computer software
Divisions:	Faculty of Engineering > Department of Informatic
Depositing User:	MOHAMMAD FARID NAUFAL
Date Deposited:	29 Aug 2022 01:58
Last Modified:	09 Jan 2023 07:49
URI:	http://repository.ubaya.ac.id/id/eprint/42403

Actions (login required)

View Item