American Sign Language Alphabet Recognition using Leap Motion Controller
Recognition of American Sign Language (ASL) alphabet not only could bring benefits to the ASL users, but also could provide solutions for natural human-computer/robot interactions in many applications. In this paper, we propose a method for ASL alphabet recognition with use of a Leap Motion Controller (LMC). The skeleton data from the native LMC API is transformed by a skeleton module into a vector of the angle features. Meanwhile, two raw infrared-radiation (IR) images are captured and each of them is fed into a vision module using a Convolutional Neural Network (CNN) for visual feature extraction, which results in two feature vectors. Those three feature vectors are then fed into a fusion neural network to output the predicted label. An ASL alphabet dataset is established, on which the proposed model is evaluated. The results show that our proposed method achieves the prediction accuracies of 80.1% and 99.7% in the leave-one-out and the half-half experiments, respectively.
W. Tao et al., "American Sign Language Alphabet Recognition using Leap Motion Controller," Proceedings of the 2018 Institute of Industrial and Systems Engineers Annual Conference and Expo (2018, Orlando, FL), pp. 599-604, Institute of Industrial and Systems Engineers (IISE), May 2018.
2018 Institute of Industrial and Systems Engineers Annual Conference and Expo, IISE 2018 (2018: May 19-22, Orlando, FL)
Mechanical and Aerospace Engineering
Intelligent Systems Center
Keywords and Phrases
American Sign Language; Convolutional Neural Network; Deep learning; Leap Motion Controller; Machine learning
Article - Conference proceedings
© 2018 Institute of Industrial and Systems Engineers (IISE), All rights reserved.
01 May 2018