DM-HAP: Diffusion model for accurate hand pose prediction

Abstract

Forecasting hand poses is a challenging task due to inherent uncertainties, occlusions, and inaccuracies in 3D pose estimation. Diffusion models provide a promising direction for predicting precise 3D hand poses under noisy conditions. In this work, we introduce Dual-diffusion, an innovative framework for the precise prediction of future hand poses. Our approach leverages the strengths of diffusion models by framing hand pose forecasting as a reverse diffusion process, effectively addressing the complexities of noisy hand joint movements and their subtleties. To enhance the learning of hand pose representations, we propose a unique neural architecture that simultaneously captures both local and global features. This is achieved through the deployment of Global and Local Diffusion (GLD) blocks within our network, which facilitate the exchange of information between local and global features. This diffusion-based interaction enables the integration of global hand actions and local finger actions, leading to a more powerful representation learning approach. We evaluate the effectiveness of our Dual-diffusion method on three public 3D hand pose estimation datasets (MSRA, F-PHAB, and BigHand2.2M), and our approach outperforms previous methods. Show more