Computer Vision and Machine Learning for Biomechanics Applications : Human Detection, Pose and Shape Estimation and Tracking in Unconstrained Environment From Uncalibrated Images, Videos and Depth

Abstract

Motivation. In Biomechanics, musculoskeletal models yield information that cannot be non-invasively obtained by direct measurement based on skeletal kinematics. Unsatisfactorily, obtaining accurate skeletal kinematics is limited to either user manual labelling or marker-based motion capture systems (MoCaps) that are limited by expansive infrastructure, environmental conditions, obtrusive markers causing movement impediment and occlusion errors. Moreover, they cannot yield surface geometry that is critical for many biomechanical applications. To advance the state of knowledge, real-time user-free acquisition of individualised pose and surface geometry is currently needed, and motivates our work on this thesis.

Aims. The goals of this dissertation are; 1) to explore how advances in computer vision and machine learning algorithms can be levered to provide the necessary framework for in-natura acquisition of skeletal kinematics, 2) in a challenge to the traditional biomechanics modelling reliance on skeletal pose only, explore how computer vision algorithms can be used to develop shape recovery framework, 3) to demonstrate the potential of human detection, tracking, pose estimation and surface recovery techniques to address open problems in biomechanics.

Contributions. We demonstrate skeletal pose estimation from monocular images in challenging environments under a discriminative pictorial structure framework. We extend the flexible part based approach to explicitly model human-object interaction. Our empirical performance results show that our proposed extension to the technique improves pose estimation. Further, we develop a hybrid framework for human detection and shape recovery using a discriminative deformable part based model for detection with a learnt shape and appearance model priors for shape recovery from monocular images. We also develop a real time framework for simultaneous activity recognition, pose estimation and shape recovery using information from a structured light sensor. For a demonstrator application, we develop a theoretical model that uses the recovered shape to solve downstream open questions in biomechanics. Finally, we develop object detection and tracking in a particularly challenging environment from image sequences that include rapid shot and view transition using complementary trained discriminative classifiers. We apply our techniques to the human ambulatory modalities of cycling and kayaking because they are common in both the clinical and sports biomechanics settings, but are rarely studied because they present unique challenges. Specifically, many applied problems relating to those modalities remain open due to absence of robust markerless motion capture that can recover skeletal kinematics and surface geometry in-natura.

Impact. The developed methods can subsequently provide new insights into open applied problems, such as enhance the understanding of bluff bodies, specifically cycling, aerodynamics, and kayaking performance. More importantly, we believe that from a higher level standpoint, our full-body human shape modelling and surface recovery represents a significant paradigm shift in biomechanical modelling, which traditionally relies on skeletal pose only. The knowledge gained is intended to form the foundation for the development of evidence-based decision support tools for diagnosis and treatment through enhanced understanding of human motion. We envision that these methods will have a transformative effect on the field of biomechanics, analogously to the effect of medical imaging on the field of medicine.