Combination of Depth and Texture Descriptors for Gesture Recognition

Automatic hand gesture recognition is a challenging problem that is attaining a growing interest due to its applications in natural interfaces for human-machine interaction, automatic sign-language recognition, computer gaming, robotics and healthcare. This chapter briefly reviews existing approaches for automatic hand gesture recognition and proposes a novel system exploiting together color and depth data. The proposed approach is based on a set of four descriptors extracted from the depth map and three texture descriptors extracted from the 2D image, while the classification is performed by an ensemble of support vector machines and decision trees. A main novelty for feature extraction is a method based on the histogram of gradients used for describing the curvature image obtained from the depth map. Another novelty is the evaluation of different colorimetric spaces for improving the recognition performance of the texture descriptors: the best performance is obtained using the lightness band of the L*c*h* color space. In the experimental Section the performances of different "stand-alone" descriptors are firstly compared and their correlation is analyzed for assessing their complementarity, and eventually the advantage gained by their fusion is demonstrated by the Wilcoxon Signed-Rank test.