Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

CONTINUOUS HAND GESTURE SPOTTING AND CLASSIFICATION USING 3D FINGER JOINTS INFORMATION

Abstract: 

This paper presents a novel approach for continuous dynamic hand gesture recognition for RGB video input. Our approach contains two main modules. Firstly, in the gesture spotting module, the video sequence with continuous gestures are pre-segmented into isolated gestures. Secondly, the gesture classification module classifies the segmented gestures. In the gesture spotting module, the motion of the hand palm and finger movements are fed into Bidirectional Long Short-Term Memory (Bi-LSTM) network for gesture spotting purpose. In the gesture classification module, two 3D Convolution Neural Networks (3D_CNN) and one Long Short-Term Memory (LSTM) network are combined to efficiently utilize the combination of multiple data channels such as RGB, Optical Flow and 3D key pose positions. The promising performance of our approach is obtained by experiments conducted on three publish datasets – Chalearn LAP ConGD dataset, 20BN-Jester, and NVIDIA Dynamic Hand gesture Dataset. Our approach achieves mean Jaccard Index of 0.5535, which outperforms the state-of-the-art methods on Chalearn LAP ConGD challenge.

up
0 users have voted:

Paper Details

Authors:
Nguyen Ngoc Hoang, Guee-Sang Lee, Soo-Hyung Kim, Hyung-Jeong Yang
Submitted On:
17 September 2019 - 2:55am
Short Link:
Type:
Poster
Event:
Presenter's Name:
GUEESANG LEE
Paper Code:
ICIP 2019 Paper #1745
Document Year:
2019
Cite

Document Files

hand gesture spotting and recognition

(11)

Subscribe

[1] Nguyen Ngoc Hoang, Guee-Sang Lee, Soo-Hyung Kim, Hyung-Jeong Yang, "CONTINUOUS HAND GESTURE SPOTTING AND CLASSIFICATION USING 3D FINGER JOINTS INFORMATION", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4652. Accessed: Oct. 20, 2019.
@article{4652-19,
url = {http://sigport.org/4652},
author = {Nguyen Ngoc Hoang; Guee-Sang Lee; Soo-Hyung Kim; Hyung-Jeong Yang },
publisher = {IEEE SigPort},
title = {CONTINUOUS HAND GESTURE SPOTTING AND CLASSIFICATION USING 3D FINGER JOINTS INFORMATION},
year = {2019} }
TY - EJOUR
T1 - CONTINUOUS HAND GESTURE SPOTTING AND CLASSIFICATION USING 3D FINGER JOINTS INFORMATION
AU - Nguyen Ngoc Hoang; Guee-Sang Lee; Soo-Hyung Kim; Hyung-Jeong Yang
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4652
ER -
Nguyen Ngoc Hoang, Guee-Sang Lee, Soo-Hyung Kim, Hyung-Jeong Yang. (2019). CONTINUOUS HAND GESTURE SPOTTING AND CLASSIFICATION USING 3D FINGER JOINTS INFORMATION. IEEE SigPort. http://sigport.org/4652
Nguyen Ngoc Hoang, Guee-Sang Lee, Soo-Hyung Kim, Hyung-Jeong Yang, 2019. CONTINUOUS HAND GESTURE SPOTTING AND CLASSIFICATION USING 3D FINGER JOINTS INFORMATION. Available at: http://sigport.org/4652.
Nguyen Ngoc Hoang, Guee-Sang Lee, Soo-Hyung Kim, Hyung-Jeong Yang. (2019). "CONTINUOUS HAND GESTURE SPOTTING AND CLASSIFICATION USING 3D FINGER JOINTS INFORMATION." Web.
1. Nguyen Ngoc Hoang, Guee-Sang Lee, Soo-Hyung Kim, Hyung-Jeong Yang. CONTINUOUS HAND GESTURE SPOTTING AND CLASSIFICATION USING 3D FINGER JOINTS INFORMATION [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4652