American Sign Language Alphabet Recognition using Microsoft Kinect

Abstract

American Sign Language (ASL) alphabet recognition using marker-less vision sensors is a challenging task due to the complexity of ASL alphabet signs, self-occlusion of the hand, and limited resolution of the sensors. This paper describes a new method for ASL alphabet recognition using a low-cost depth camera, which is Microsoft's Kinect. A segmented hand configuration is first obtained by using a depth contrast feature based per-pixel classification algorithm. Then, a hierarchical mode-seeking method is developed and implemented to localize hand joint positions under kinematic constraints. Finally, a Random Forest (RF) classifier is built to recognize ASL signs using the joint angles. To validate the performance of this method, we used a publicly available dataset from Surrey University. The results have shown that our method can achieve above 90% accuracy in recognizing 24 static ASL alphabet signs, which is significantly higher in comparison to the previous benchmarks.

Meeting Name

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2015: Jun. 7-12, Boston, MA)

Department(s)

Mechanical and Aerospace Engineering

Second Department

Computer Science

Keywords and Phrases

Computational Linguistics; Computer Vision; Decision Trees; Gesture Recognition; Joints ( Structural Components); Kinematics; Probability Distributions; Accuracy; Alphabet Recognition; American Sign Language; Hand Configuration; Kinematic Constraints; Microsoft's Kinect; Pixel Classification; Thumb; Pattern Recognition

International Standard Book Number (ISBN)

978-1467367592

International Standard Serial Number (ISSN)

2160-7508

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2015 IEEE Computer Society, All rights reserved.

Publication Date

01 Oct 2015

Link to Full Text

Share

 
COinS