On Motion Estimation Problems in Computer Vision

Abstract

Motion estimation is one of the fundamental problems in computer vision. It has broad applications in the fields of robot navigation, mixed and augmented reality, visual tracking, image and video processing, intelligent transportation systems and so on. Up until now, motion estimation is far from a solved problem, and it is still one of the active research topics in and beyond the computer vision community. This thesis is dedicated to both camera motion estimation -- including motion estimation for 3D and 2D cameras -- and dense image motion for color images. We push the limits of the state of the art in various aspects such as optimality, robustness, accuracy and flexibility. The main contributions are summarized as follows.

First, a globally optimal 3D point cloud registration algorithm is proposed and applied to motion estimation of 3D imaging devices. Based on Branch-and-Bound (BnB) optimization, we optimally solve the registration problem defined in Iterative Closest Point (ICP). The registration error bounds are derived by exploiting the structure of the SE(3) geometry. Other techniques such as the nested BnB and the integration with ICP are also developed to achieve efficient registration. Experiments demonstrate that the proposed method is able to guarantee the optimality, and can be well applied in estimating the global or relative motion of 3D imaging devices such as 3D scanners or depth sensors.

Second, a globally optimal inlier-set maximization algorithm is proposed for color camera motion estimation. We use BnB to seek for the optimal motion which gives rise to the maximal inlier set under a geometric error. An explicit, geometrically meaningful relative pose parameterization -- a 5D direct product space of a solid 2D disk and a solid 3D ball -- is proposed, and efficient, closed-form bounding functions of inlier set cardinality are derived to facilitate the 5D BnB search. Experiments on both synthetic data and real images confirm the efficacy of the proposed method.

Third, a scene constraint based method for relative pose estimation between a 2D color camera and a 3D sensor is developed. We formulate the relative pose estimation as a 2D-3D registration problem minimizing the geometric errors from the known scene constraints. Our method takes only a single pair of color and depth images as input, and is correspondence-free. In addition, a new single-view 3D reconstruction algorithm is proposed for obtaining initial solutions. The experiments show that the method is both flexible and effective, producing accurate relative pose estimates and high-quality color-depth image registration results.

Fourth, a highly-accurate optical flow estimation algorithm based on piecewise parametric motion model is proposed. It fits a flow field piecewise to a variety of parametric models where the domain of each piece (i.e., shape, position and size) and its model parameters are determined adaptively, while at the same time maintaining a global inter-piece flow continuity constraint. The energy function takes into account both the piecewise constant model assumption and the flow field continuity constraint, enabling the proposed algorithm to effectively handle both homogeneous motions and complex motions. The experiments on three public optical flow benchmarks show that the proposed algorithm achieves top-tier performances.

At last, we propose a robust algorithm for optical flow estimation in the presence of transparency or reflection. It deals with a challenging, frequently encountered, yet not properly investigated problem in optical flow estimation: the input two frames contain one background layer of the scene and one distracting, possibly moving layer due to transparency or reflection. The proposed algorithm performs both optical flow estimation and image layer separation. It exploits a generalized double-layer brightness consistency constraint connecting these two tasks, and utilizes the priors for both of them. The experiments on synthetic and real images confirm its efficacy.