American Sign Language Alphabet Recognition using Microsoft Kinect


American Sign Language (ASL) alphabet recognition using marker-less vision sensors is a challenging task due to the complexity of ASL alphabet signs, self-occlusion of the hand, and limited resolution of the sensors. This paper describes a new method for ASL alphabet recognition using a low-cost depth camera, which is Microsoft's Kinect. A segmented hand configuration is first obtained by using a depth contrast feature based per-pixel classification algorithm. Then, a hierarchical mode-seeking method is developed and implemented to localize hand joint positions under kinematic constraints. Finally, a Random Forest (RF) classifier is built to recognize ASL signs using the joint angles. To validate the performance of this method, we used a publicly available dataset from Surrey University. The results have shown that our method can achieve above 90% accuracy in recognizing 24 static ASL alphabet signs, which is significantly higher in comparison to the previous benchmarks.

Meeting Name

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2015: Jun. 7-12, Boston, MA)


Mechanical and Aerospace Engineering

Second Department

Computer Science

Keywords and Phrases

Computational Linguistics; Computer Vision; Decision Trees; Gesture Recognition; Joints ( Structural Components); Kinematics; Probability Distributions; Accuracy; Alphabet Recognition; American Sign Language; Hand Configuration; Kinematic Constraints; Microsoft's Kinect; Pixel Classification; Thumb; Pattern Recognition

International Standard Book Number (ISBN)


International Standard Serial Number (ISSN)


Document Type

Article - Conference proceedings

Document Version


File Type





© 2015 IEEE Computer Society, All rights reserved.

Publication Date

01 Oct 2015