3D Hand Pose Estimation Using Convolutional Neural Networks

June 6, 2018
Qi Ye | Imperial College London

3D hand pose is challenging due to complicated variations caused by high Degree of Freedom articulations, multiple viewpoints, self-similar parts, severe self-occlusions, different shapes, and sizes. My Ph.D. work tackles 1) the multiple viewpoints and complex articulations of hand pose estimation by decomposing and transforming the input and output space by spatial transformations following the hand structure; 2) the multi-modality of the locations for occluded hand joints by a hierarchical mixture density deep network which leverages the state-of-the-art hand pose estimators based on Convolutional Neural Networks to facilitate feature learning while models the multiple modes in a two-level hierarchy to reconcile single-valued (for visible joints) and multi-valued (for occluded joints) mapping in its output; 3) the lack of complete labeled real hand datasets by a tracking system with six 6D magnetic sensors and inverse kinematics to automatically obtain 21-joints hand pose annotations of depth maps.

Speaker Details

Qi Ye is a Ph.D. candidate at Computer Vision and Learning Lab, Imperial College London, under the supervision of Dr. Tae-Kyun Kim. She is interested in human-computer interaction, particularly understanding and control in hand-object interaction scenarios. Her Ph.D. research focuses on 3D hand pose estimation, and these works have been published in ECCV, CVPR, and PAMI. She also co-organised the ICCV2017 Hand Workshop and the 2017 Hands in the Million Challenge. Before studying at Imperial, she received her M. Eng. degree in Electronic Engineering from Tsinghua University in 2014 and her B. Eng. degree in Electronic Engineering from Beijing Normal University in 2011 in China.