Deng, Huizhong
Description
Non-rigid Structure-from-Motion (NRSfM) is an active research eld in computer
vision. The task of NRSfM is to simultaneously recover camera motion and 3D
structure from 2D tracks of a deformable object. This problem is generally categorized
into sparse and dense cases in terms of scale, where sparse NRSfM deals with
a few feature tracks and dense NRSfM recovers the 3D position of each pixel in
an image
ow. As NRSfM is essentially an under-constrained problem, recent research
has...[Show more] focused on enforcing priors to reliably solve the problem. In this thesis,
we propose a shape clustering method for sparse NRSfM and a spatial-temporal
constraint for dense NRSfM.
For sparse NRSfM, we rst revisit the concept of \reconstructability", which
indicates the possibility of reconstructing a 3D shape, given 2D feature tracks and
camera motion. We give an extension to it and de ne \reconstructability" from
3D shape complexity and motion complexity. To increase global reconstructability,
we then propose an iterative shape clustering method to divide a sequence into
several sub-sequences, thus decreasing the shape complexity of each sub-sequence,
which is much easier to solve individually. Our method aims at solving the longterm,
complex motions, which have been a di cult task for previous methods.
Experimental results show that our method outperforms the current state-of-theart
methods by a margin, thus pushing the limit of sparse NRSfM.
For dense NRSfM, we rst revisit the temporal smoothness utilized in sparse
NRSfM and demonstrate that it can be employed for dense case directly. Secondly,
we propose a spatial smoothness constraint by enforcing a Laplacian lter to the
shape matrix. Finally, to handle real world noise and outliers in measurements,
we robustify the data term by using the L1 norm. Our method gives a simple
yet elegant convex least-squares optimization, which can be e ectively solved by
gradient descent. Experimental results on both synthetic and real images show
that the proposed method achieves state-of-the-art performance in dense NRSfM.
Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.